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Abstract 

We design a dynamic rate scheduling policy of Markov type via the solution (a so- 
cial optimal Nash equilibrium point) to a utility-maximization problem over a randomly 
evolving capacity set for a class of generalized processor-sharing queues living in a random 
environment, whose job arrivals to each queue follow a doubly stochastic renewal process 
(DSRP). Both the random environment and the random arrival rate of each DSRP are 
driven by a finite state continuous time Markov chain (FS-CTMC). Whereas the schedul- 
ing policy optimizes in a greedy fashion with respect to each queue and environmental 
state and since the closed-form solution for the performance of such a queueing system 
under the policy is difficult to obtain, we establish a reflecting diffusion with regime- 
switching (RDRS) model for its measures of performance and justify its asymptotic op- 
timality through deriving the stochastic fluid and diffusion limits for the corresponding 
system under heavy traffic and identifying a cost function related to the utility function, 
which is minimized through minimizing the workload process in the diffusion limit. More 
importantly, our queueing model includes both J-user multi-input multi-output (MIMO) 
multiple access channel (MAC) and broadcast channel (BC) with cooperation and admis- 
sion control as special cases. In these wireless systems, data from the J users in the MAC 
or data to the J users in the BC is transmitted over a common channel that is fading 
according to the FS-CTMC. The J-user capacity region for the MAC or the BC is a 
set-valued stochastic process that switches with the FS-CTMC fading. In any particular 
channel state, we show that each of the J-user capacity regions is a convex set bounded 
by a number of linear or smooth curved facets. The random arrival rate to each user 
for these systems is designed to switch with the FS-CTMC fading via admission con- 
trol. At the transmit end, packets to each user are queued and served under the policy. 
Therefore our queueing model can perfectly match the dynamics of these wireless systems. 
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1 Introduction 

In the current cellular systems, each base station is considered as a separate entity with no 
cooperation among base stations, infrastructure cooperation among base stations has been 
proposed in the hterature such as [1, 33, 48], which is to consider the base stations as one 
end of a MIMO system that has received a great deal of attention as a method to achieve 
high data rates over wireless links. Thus, in this paper, we study a J-user MIMO MAC 
uplink system and a J-user MIMO BC downlink system. Both of them can be seen as a 
cellular system with multiple users and multiple cooperating base station antennas: either 
multiple cooperating base stations each with a single antenna or a single-cell cellular system 
with a multi-antenna base station or a combination thereof. In the MAC or the BC, data is 
buffered at the transmit end and the channel is time-varying due to multipath fading, which 
is a typical feature of wireless channel and brings additional complexity for system design 
and performance analysis. We suppose that the fading process is a FS-CTMC whose discrete 
time version is widely used in modeling wireless channels (see, e.g., [49, 43, 48, 22], and 
references therein). Therefore, the J-user capacity regions of the MAC and the BC are both 
time- varying set-valued stochastic processes driven by the FS-CTMC and in each state of the 
Markov chain, it is well known that one can obtain the improved capacity by cooperation, 
e.g., the sum of the rates at which data can be served for the J users is greater than the 
single-user capacity for any user (see, e.g., [3]). Moreover, due to the impact of the random 
environmental fading factor and the cooperated design, the service rates of the corresponding 
queueing system for the J users in the MAC or in the BC are also random processes driven 
by the FS-CTMC. 

So, motivated by the above observations, we consider a type of generalized processor- 
sharing queues living in a random environment, whose job arrivals to each queue follow a 
DSRP. Both the random environment and the random arrival rate of each DSRP are driven 
by a FS-CTMC. Presently, for such a queueing system, it is not known how to choose a 
reasonable online rate scheduling policy to minimize the average delay for a given load and 
exact solutions for average delay are not available even for many simple policies, which implies 
that any meaningful comparison has to be done by simulations. Therefore, to make the gap 
between the dynamic rate scheduling and the performance optimization for the system be 
filled to some extent, we design a dynamic rate scheduling policy of Markov type via the 
solution (a social optimal Nash equilibrium point) to an optimization problem that maximizes 
a general utility function over each of the randomly evolving capacity regions through the 
Karush-Kidm- Tucker (KKT) optimality conditions (see, e.g., [35]). Moreover, to overcome 
the intractability of performance evaluation for the system under the designed policy, we 
develop stochastic fluid and diffusion models through suitable scaling of time and space and 
justifying related limit theorems for a heavily loaded queueing system operating under this 
policy. The limit models for queue lengths (or workloads) are respectively a random process 
driven by the FS-CTMC and a RDRS (i.e., a reflecting stochastic differential equation (SDE) 
with regime-switching). In addition, we identify a cost function related to the utility function. 
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which is minimized through minimizing the workload process in the diffusion hmit and hence 
provides a useful means in illustrating our policy to be asymptotically optimal. 

Finally, in order to incorporate the J-user MIMO MAC and MIMO BC into our general 
queueing framework, we justify that the J-user capacity region for the MAC or the BC in 
any particular channel state is a convex set formed by a number of linear or smooth curved 
facets through applying the method of convex optimization, the implicit function theorem, 
and the duality of capacity regions between the MAC and the BC. Moreover, to realize the 
DSRP in the MAC or in the BC, we adopt a cross-layer design methodology to switch the 
arrival rates with the FS-CTMC channel fading process according to the current channel 
state information (CSI) through admission control. 

Literature Review 

The randomly evolving capacity region used in designing our utility-maximization rate 
scheduling policy is a generalization of the so-called MIMO channel capacity region in the 
Shannon theoretic sense. For a single-user time-invariant channel, the Shannon capacity is 
defined as the maximum mutual information between input and output, which is shown by 
Shannon's capacity theorem to be the maximum data rate that can be transmitted over 
the channel with arbitrarily small error probability. For a J-user time-invariant MIMO 
channel, the corresponding capacity region is a J-dimensional set of all rate vectors (ci, cj)' 
simultaneously achievable by all J users. In particular, the region for the Gaussian MAC is a 
convex set that is the union of rate regions corresponding to every product input distribution 
satisfying the user-by-user power constraints (see, e.g., [14], [12], [56], [25]). The Gaussian 
BC differs from the Gaussian MAC in two fundamental aspects (see, e.g., [31]). In the MAC, 
each transmitter has an individual power constraint, whereas in the BC there is only a single 
power constraint on the transmitter. Moreover, signal and interference come from different 
transmitters in the MAC and are multiplied by different channels gains (known as the near- 
far effect) before being received, whereas in the BC, the entire received signal comes from 
the same source and therefore has the same channel gain. Nevertheless, the capacity region 
for the Gaussian BC can be obtained through the duality between the Gaussian MAC and 
the Gaussian BC (see, e.g., [31] and [25]), i.e., it is the convex hull of the union over the set 
of capacity regions of the dual Gaussian MACs such that the total MAC power is the same 
as the power in the BC. Moreover, the authors in [34] provide an analytical and numerical 
characterization in terms of the shape of the capacity boundaries for both the MAC and the 
BC. 

However, in both the Gaussian MAC and the Gaussian BC, the exact characterization 
concerning piecewise smoothness of the capacity boundaries is not available until now, which 
motivates us to give more accurate analysis about the capacity region in order to apply our 
utility maximization rate scheduling algorithm to these wireless systems. In addition, when 
the J-user MIMO channels arc stochastic and time- varying fading ones, the capacity regions 
have multiple definitions (see, e.g., [25]). Nevertheless, to capture the exact capacity region 
at each time instant for the MAC or the BC, we consider the capacity regions as a set- valued 
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stochastic process evolving with the FS-CTMC rather than think of it as a fixed one in an 
average sense such as an ergodic capacity region (see, e.g., [25]). 

Concerning the scheduhng algorithms, the authors in [1, 3, 4] considered a quasi-static 
downlink channel, where the channel is assumed to be fixed for all transmissions over the 
period of interest. In this case, the FS-CTMC and the random packet arrival rates assumed 
in the current paper reduce to constants, and moreover, without considering utility and cost 
optimization, the authors in [1, 3, 4] designed a simple rate scheduling policy of Markov 
type, which was shown to be throughput-optimal for a fixed convex capacity region in [1] 
and a limit theorem was proved to justify the diffusion approximation of the queue length 
process for a heavily loaded system operating under their policy with two users in [3] and 
with multiple users in [4] . Their approximating model is a RBM living in the two-dimensional 
positive quadrant or in the general-dimensional positive orthant. 

In the studies of [43, 41, 22], some scheduling policies were considered for certain heavily 
loaded wireless systems with finite state discrete time Markov fading process. In particu- 
lar, a Max;Weight scheduling policy was considered in [43] for a generalized switch and it was 
shown that the workload process converges to a one-dimensional RBM and MaxWeight policy 
asymptotically minimizes the workload under certain conditions. Moreover, an exponential 
scheduling rule was designed for wireless channels in [41] and for a generalized switch in [22], 
which was proved to be throughput-optimal and under which, the similar results concern- 
ing the workload process were obtained and justified as in [43]. In addition, [54] designed 
a utility-maximizing resource allocation policy for a class of stochastic networks with con- 
current occupancy of resources and established its asymptotic optimality for the associated 
heavily loaded queueing system. Their policy covers the generalized c^u-rule in [36] and the 
MaxWeight policy in [43] as special cases. 

The differences between the current study and those in [43, 41, 22, 54] are in three aspects 
as follows. 

First, their scheduling policies in [43, 41, 22, 54] depend only on a fixed capacity region 
that is a convex polyhedral and ours depends on a time-varying and stochastic evolving 
capacity region process (a random environment) that, at each time instant, is a more general 
convex region rather than a convex polyhedral. 

Second, the rates of packet arrivals to the J users are random processes rather than a 
constant as used in [43, 41, 22, 54]. Hence our input traffic to each user is a DSRP whose 
particular case is the well-known doubly stochastic Poisson process (see, e.g., [8]) that is 
widely used to model voice, video and data source traffics in telecommunication systems and 
is called Markovian modulated Poisson process (MMPP) or ON/OFF source (see, e.g., [30], 
[37], [44], [20]) and [21])). 

Third, our discussion is based on a continuous time horizon rather than a discrete one 
as in [43, 41, 22]. Therefore our vector-valued random service rate process depends on the 
FS-CTMC whose holding time at each environmental state has an important impact on the 
limiting processes, e.g., the limiting fluid model is a random process driven by the FS-CTMC 
rather than a deterministic function of time and the limiting diffusion model is a more general 
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RDRS rather than a RBM as derived in [43, 41, 22]. If one wants to directly generaUze the 
studies in [43, 41, 22] to the corresponding ones in a discrete time random environment, a 
geometric distribution may be imposed on the holding time at each environmental state. 

Finally, without considering optimal dynamic scheduling with utility/cost and perfor- 
mance optimizations as the goals. CTMCs have been used to model the random environ- 
ments in the studies of some queueing systems under certain static service disciplines, see, 
e.g., [13] and references therein for more details. 

The rest of the paper is organized as follows. In Section 2, we introduce our generalized 
processor-sharing queues under random environment and design our optimal rate scheduling 
policy. In Section 3, we introduce our heavy traffic condition and present our main asymptotic 
optimality theorem. In Section 4, wc illustrate the usages of our optimal policy and our main 
results in the J-uscr MIMO uplink and downlink wireless channels and present the associated 
results concerning the pieccwise smoothness of capacity boundaries of the J-user MIMO MAC 
and MIMO BC. In Sections 5-6, we prove our main theorem and associated lemmas. 

2 Optimizing Processor-Sharing Queues under Random Environment 
2.1 Primitive Data 

The queueing system under consideration is a type of generalized processor-sharing queues 

that live in a random environment evolving according to a stationary FS-CTMC a = {a{t),t G 
[0,oo)}, which takes value in a finite state space K, = {1,...,K} with generator matrix 
G = (gu) (ij G /C) and 



where 7(z) is the holding rate for the chain in an environmental state i G {1,...,K} and 
Q = ilii) is the transition matrix of its embedded discrete time Markov chain (see, e.g., [39]). 
Moreover, the queueing system has J queues in parallel, which correspond to J users for a 
given positive integer J. Each queue that is of infinite buffer capacity buffers packets (jobs) 
arrived for a given user. The queues can be served simultaneously by a single server with rate 
allocation vector c{t) = (ci(i), cj(i))' that takes values in a time- varying and randomly 
evolving capacity set TZ{a{t)). 

Concretely, for each state i G /C, TZ{i) is a convex set that contains the origin and has 
L (> J) boundary pieces of which J are ( J — l)-dimensional linear facets along the coordinate 
axes while the remaining ones are in the interior of and form the so-called capacity surface 
denoted by 0{i), which consists of B = L — J {> 0) linear or smooth curved facets hk{c,i) 
on R-^foT keU = {1, 2, B}, i.e.. 



(2.1) 




(2.2) 
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Moreover, if we let Cu denote the sum capacity upper bound for the capacity region, then 
the facet in the center of the capacity surface is linear and can be expressed by 

J 

(2.3) hk^j (c, i) =Y1 - Cu 

i=i 

where ku '^s the index corresponding to Cu. Moreover, we suppose that any one of the 
J hnear facets along the coordinate axes forms a (J — l)-user capacity region corresponding 
to a particular group of J — 1 users who are the only users in the systems. Similarly, we can 
define the (J — j)-user capacity region for each j G {2, J — 1}. Examples of such capacity 
sets in two- and three-dimensional spaces for a particular state i € /C are shown in Figures 1 
and 2. 




Figure 1: A 2-user capacity set in the 2-diniensional space in a particular environmental state 



t 

I 




Figure 2: A 3-user capacity set in the S-dimensional space in a particular environmental state 

In addition, we suppose that the system starts empty and that there is a J-dimensional 
packet arrival process A = {A{t) = (^i(t), ^j(f))', f > 0}, where Aj{t) with j ^ J and 
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t > is the number of packets arrived to the jth queue during (0, t] and the prime denotes 
the transpose of a vector or a matrix. For each j G J', Aj{-) is assumed to be a DSRP with 
random arrival rate process Xj{a{-)) and squared coefficient of variation process a^(a(-)) G 
(0,00). The packet interarrival times are assumed to be i.i.d. during the time interval 
corresponding to a specific environmental state i £ JC. Moreover, let {uj{k),k = 1,2,...} 
denote the sequence of times between the arrivals of the (k — l)th and the kth packets to 
the j'th queue and let {vj{k),k = 1,2,...} denote the sequence of packet lengths (in bits) 
for the successive arrivals to queue j, which is assumed to be a sequence of strictly positive 
i.i.d. random variables with average packet length l/nj € (0, 00) and squared coefficient of 
variation /3? G (0, 00). In addition, we suppose that all interarrival and service time processes 
are mutually (conditionally) independent when the environmental state is fixed. For each 
J G i7 and each nonnegative constant h (in bits), we use Sj{-) to denote the renewal counting 
process associated with {vj{k),k = 1,2,...}, i.e.. 



The rcasonability about the DSRP assumption on the packet arrivals and about the i.i.d. 
assumption on the packet sizes in a communication system is due to the large-scale computer 
experiments and statistical analysis conducted by Bell Labs scientists [10] , and recent findings 



2.2 A Utility-Maximization Scheduling Algorithm and Queueing Dynamics 

First of all, we remark that the service discipline used in this paper is the so-called head of 
line discipline under which the service goes to the packet at the head of the line for a serving 
queue where packets are stored in the order of their arrivals. The service rates arc determined 
by a function of the environmental state and the number of packets in each of the queues. 
At each state i € IC and for a given queue length vector q = {qi, ■■■,qjy , let A{q,i) denote 
the corresponding rate vector (in bps) of serving the J queues, which is a solution of the 
following utility maximization problem 



where c = (ci,...,cj)' is a J-dimensional vector and Uj{qj,Cj) for each j G ^ is a util- 
ity function defined on R"^, which is second-order differentiable and satisfies the following 
conditions 



(2.7) Uj{qj,Cj) = <^j{qj)'^{cj) is strictly increasing and concave in Cj for each g^- > 0, 



(2.4) 




by [20]) and [21]). 



(2.5) 




(2.6) [/,(0,c,) = 0. 



(2.8) 



dUj{qj,Cj) 



is strictly increasing in qj >0, 



dcj 



(2.9) 



dUj{Q,Cj) 



and lim 



dUj{qj,Cj) 



+00 for each Cj > 0. 
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(2.10) A{q,i) = 



A{q,i) 



Due to condition (2.7), we know that there must exist an optimal solution in the following 
form for a given q, 

\Q{ki,-,km)(^q^j^y [f q e Q{ki,...,km) and a given me J, 
(0,0,. ..,0) if g = 0, 

where kj e J for each j e {l,...,m} and kj ^ ki if j ^ I. Moreover, Qiki, km) denotes 
the set of all q e that have exactly m components qk^ [j e {1, m}) to be zero, and the 
components of A^('^i' - ''^'")(g, i)' corresponding to kj {j G v7\{l, rn}) consist of the optimal 
solution to (2.5) with the capacity region TZ{i) replaced by the corresponding (J — m)-user 
capacity region and all other components of A2('^'i' ' '^™)(g, i)' are zero. For example, when 
there are only two users in the system, (2.10) is of the following form, 

{ci{q,i),C2{q,i))' if g > 0, 

{c\{i),Q) if (?i >0, (?2 = 0, 

(0,c|(i)) if gi = 0, 52 >0, 

^ (0,0) if gi = 52 = 0. 

Remark 2.1 The optimal solution to (2.5) may not he unique when qj = for some j G J^, 
however, if Aj{q,i) > with qj = for some j G J^, we can reset Aj{q,i) to zero without 
violating the constraints or decreasing the objective value (referred to (2.7)). Hence, whenever 
the solution to (2.5) is concerned, we will always suppose that (2.10) is true. Moreover, for 
each q > (and similarly, for a lower dimensional case), it follows from (2.7) that every 
point on the capacity surface defined in (2.2) is a Nash equilibrium point to a concave game 
in the sense of [40] and therefore the solution to (2.5) is a social optimal Nash equilibrium 
point to the concave game. 

In addition, we assume that {Uj{qj,Cj),j G J'} satisfies the so-called radial homogeneity 
condition, i.e., for any scalar a > 0, each q > and each i e fC, its maximizer satisfies 

(2.11) Aj{aq,i) = Aj{q,i). 

Interested readers are referred to [54] for numerous examples of the utility function that 
satisfies conditions (2.6)-(2.9) and (2.11), such as, the so-called proportional fair allocation, 
minimal delay allocation, and {/3, a)-proportionally fair allocation, which are widely used in 
communication protocols. 

2.3 The Dual Cost Minimization Problem 

In this subsection, we consider the following cost minimization problem for each z G /C, a 
given c G 7^(i) and a given parameter w > 0, 

(2.12) mmV{q,c) 

J 



1L 

3=1 



s.t. ^^>W, 



qj > for each j e J 
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where the function V is defined by 

J 

(2.13) V{q,c)=Y.C,iq„c,) 

and Cj is the cost function associated with the utihty function Uj in (2.5), i.e., 

1 [ii dUj{u,Cj) 



(2.14) c,{q,,c,) = - r 

fin Jo 



-du. 



fij Jo dcj 

In other words, when the environment is in state i € IC, we try to identify a queue state q 
corresponding to a given c G TZ{i) and a given parameter w > STich that the total cost over 
the system is minimized and the (average) workload meets or exceeds w. 

2.4 Performance Measure Processes 

Let Q j{t) denote the queue length for the jth queue with j £ J' at each time t G [0, oo), i.e., 
(2.15) Qjit) = QjiO)+Ajit)-Djit) 

where Dj{t) is the number of packet departures from the jth queue in (0,t], i.e., Dj(t) = 
Sj{Tj{t)), where 



(2.16) TJt)= Aj{Q{s),a{s))ds 

Jo 

which denotes the cumulative amount of service (measured in bits) given to the jth queue 
up to time t. Moreover, let W{t) denote the (expected) workload at time t and Y(t) denote 
the unused capacity up to time t, i.e., 

(2.17) w{t) = j2^, Y{t) = j2{f pMs))ds-Ut)) 

where, for each i £ IC, p{i) = (pi(i), ...,pj(i))' is a given point on the capacity surface 0{i) 
and it is chosen to satisfy 

(2.18) Y^Pji^) = ™^x I ^9 1 = Cu and pi{i) = ... = pj(i). 

Here we remark that the second condition in (2.18) and the separable condition in (2.8) are 
required in proving Lemmas 5.5-5.6. However, when only a constant environment (e.g., a 
pseudo channel in a wireless system) is concerned, these two conditions can be removed. 
Obviously, 

(2.19) y(t) is non-decreasing in i > 
since, for each i > 0, we have 

(2.20) X^A,(Q(t),a(i))<Ep,(a(0)- 
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3 Main Theorem: Asymptotic Optimality 



In this section, we present the optimahty result for our scheduhng pohcy by considering 
the operation of the queueing system in the asymptotic regime where it is heavily loaded. 
Concretely, we define three sequences of diffusion-scaled processes Q^{ ), W^{-) and Y^{-) by 

(3.1, 

for each t >0 and j G J, which associate with a sequence of independent Markov processes 
{a'^{-),r G {1,2,...}}. These systems indexed by r all have the same basic structure as 
described in the last section except the arrival rates A^'(i) and the holding time rates 7''(i) for 
all i G /C, which may vary with r G {1, 2, ...} and satisfy the following heavy traffic condition 

(3.2) r (aJ(z) - Ojii) as r ^ oo, = ^ 

for each j G J, where Oj{{) G R are some constants and = ^jPj(i) are the nominal 

average packet arrival rates when the channel is in state i G IC. 

Note that, due to the heavy traffic condition in (3.2) for the rth environmental state 
process c/'i-) with rG{l,2,...}, we know that a^{r^-) and «(•) equal each other in distribution 
since they own the same generator matrix (see, e.g., the definition in pages 384-388 of [39]). 
Hence, in the sense of distribution, all of the systems indexed by r G {1,2, ...} in (3.1) share 
the same random environment over any time interval [0,t]. 

Moreover, let -B^(-) and B^{-) denote the two independent J-dimensional standard Brow- 
nian motions, and for each i G /C, let 



(3.3) A(i) = (Ai(z),...,A,/(i))', 

(3.4) p{i) = (pi(z),...,pj(z))', 

(3.5) e{i) = {9^{i),...,ej{t))', 

(3.6) r^(i) = (rf;(0)_^^_^^diag(Ai(Oa?(0,-,Aj(Oa'j(i)), 

(3.7) T^ii) = (rf,(z))^^^ = diag(Ai(0/3^...,Aj(0/33), 

(3.8) H%t) = {Hf{ty,...,H}{t)y with e denotes E or S, 

(3.9) H^{t) = [\r^^^{a{s))y'dB]{s). 

J 



In addition, let Q'''^(-) and W^'^{-) denote the diffusion-scaled queue-length and workload 
processes under an arbitrarily feasible rate scheduling policy G, e.g., a simple Markovian 
policy as studied in [4] or a policy A'^(Q''(t), a(t)) that may not be the optimal solution to 
the utility maximization problem (2.5). Then we have the following theorem. 

Theorem 3.1 Suppose <5^(0) = for all r G {1,2,...} and the heavy traffic condition (3.2) 
holds, then under the scheduling policy (2.10), we have the claims as stated in the following 



10 



two parts: 

Part A: Along r G {1, 2, ...}, the following convergence in distribution is true, 

(3.10) {Q^{-),W^{-),r{-)) {Q{-),W{-),Y{.)) 

and the limits Q{-),W{-) and Y{-) are continuous a.s., which satisfy the following RDRS 

(3.11) W{t) = X{t) + Y{s)>Q 
where 

J 

(3.12) dX{t) = Y,— iOj{a{t))dt + dHf{t) + dH^{t)) 

j=i 

Moreover, {W{-),Y{-)) is the unique solution of (3.11) with the following complementary 
property: 

1. y(o) = 0, 

2. Y{-) is non- decreasing, 

3. Y{-) can increase only at a time t G [0, oo) that W{t) = 0. 
In addition, we have 

(3.13) Q{t) = q*{W{t),p{am 

with q*{w,p{i)) being the solution to the cost minimization problem (2.12) in terms of each 
given w and i E IC. 

Part B: The workload W{-) and the cost J2j=i CjiQji')^ Pi'^i'))) ^''^^ minimal with probability 

one in the sense that, for all t > 0, 

(3.14) liminf ^^'■'^(i) > W(t), 

r-^oo 

J J 

(3.15) limmfJ2C,{Qf{t),p,iam > J2Cj{Qj{t), pM*)))- 

j=i j=i 

Remark 3.1 Corn,paring with the RBM widely studied in queueing literature, the RDRS 
model derived in (3.11) exhibits its new feature in the sense that it corresponds to a more 
realistic FS-CTMC fading process in certain applications such as in a wireless system and 
indicates that the random process a(-) is a non-ignorable random environmental factor to the 
system performance even in the limiting approximation model. From the model, we can also 
see that, when a constant environment (e.g., a quasi-static channel in a wireless system) is 
concerned, the model in (3.11) reduces to a RBM since the state process a(-) keeps a constant. 
Moreover, by the discussions in [11], [18], [17], and [26], we know that the unique solution 
(W{t),Y{t)) to (3.11) can be represented by {W ,Y) = {^{X),-^{X)), where $(•) and *(•) 
are Lipschitz continuous mappings. In addition, a RDRS is different from a conventional 
SDE since its drift and diffusion coefficients are not adapted to the filtration generated by 
the driving Brownian motions. This type of SDEs without boundary reflections has received 
a great attention in the area of financial engineering (see, e.g., [57]). 
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4 Applications to J-user MIMO Uplink and Downlink Wireless Channels 

In this section, we apply the discussions in the previous sections to a cellular system where 
base stations cooperate among noise- free infinite capacity links. We do not make any dis- 
tinction between a single-cell cellular system having multiple base-station antennas and the 
traditional cellular system with cooperating single-antenna base stations. Here, the coop- 
eration means that the base stations can perform joint beamforming and/or power control 
but there is a constraint on the total power that the base stations can share. Therefore, 
our wireless system can be considered consisting of a base station having M antennas and 
J users (mobiles), each of which has N antennas. Thus the uplink channel can be mod- 
eled as a J-user MIMO MAC and the downlink channel can be modeled as a J-user MIMO 
BC (see, e.g.. Figure 3). The channel fading is supposed to obey the stationary FS-CTMC 




Broadcast Channel Multiple Access Channel 



Figure 3: The BC and MAC channels in a particular environmental state 

a = {a{t),t G [0,oo)} that is described in the previous sections. Moreover, we suppose that 
the receive or transmit end (the cooperating base stations) has perfect CSI. For each channel 
state z € /C, we let Hj{i) [j ^ J = {1, J}) denote the downlink channel matrix from the 
base station to user j. Assuming the same channel is used on the uplink and downlink, then 
the uplink matrix of user j is -ffj(^) that is the conjugate transpose of Hj{i). 

Moreover, at the transmit end, arriving packets for each user are buffered before trans- 
mission and the rate of arrivals is a random process that switches with the FS-CTMC channel 
fading through admission control. Therefore, the processor-sharing queues presented in the 
previous section can be used to model the channel dynamics for both J-user MIMO MAC 
and J-user MIMO BC. The remaining issue is about how to characterize the MAC and BC 
capacity region processes, which is also a central topics in information theory literature. 
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4.1 The MIMO MAC Capacity Region 



In the MAC and for each channel state i E K., let Uj{i) G C^^^ be the transmitted signal 

of user j, where C^^^ denotes the N x 1 complex matrix, and let V{i) G (J^^xi (jgj^ote the 
received signal, W G C^^^^ denote the noise vector where W ~ N{0, 1) is circularly symmetric 
complex Gaussian with identity covariancc (note that the notation W here has the different 
meaning from the workload process W{t) defined in (2.17)). Then the received signal at the 
base station is equal to 

(4.1) V{i) = H^i)U'{i) + W 

where H^i) = (i), i7^(i)] and U{i) = [U{{i), ...Uj{i)] (see, e.g.. Figure 3). Moreover, 
each user j is subject to an individual power constraint Pj. The transmit covariance matrix of 
user j is defined to be = E[Uj{i)Uj (i)]. The power constraint implies that Tr(rj(i)) < 
Pj for j G i7. During the period of each channel state i G /C, it follows from [25] and [56] 
that the MAC capacity region is a J-dimensional closed convex set in R"^ = {c G R"^ : Cj > 
0,j ej}, i.e., 

(4.2) n{i)=CMAc{Pi,-,Pj,HHi)) = 



I + Y,H]{i)T,{i)Hj{t 



,V 5c J 



{rjii)>o,TT{rj{i))<Pj,j&j} 

where is a subset of J' and | • | denotes the determinant of a matrix. Moreover, every point 
in Tl{i) can be achieved by Shannon's source coding theorem and successive decoding (see, 
e.g., [24] and [25]). However, in designing a utility maximization based rate scheduling policy, 
we need to know more detailed boundary characterization of the MAC capacity region since 
it frequently relies on the KKT optimality conditions (see, e.g., [35] and [34]). Thus we have 
the following lemma. 

Lemma 4.1 For the J-user MIMO MAC and each channel state i G /C, TZ{i) contains the 
origin and has L linear or smooth curved facets with L given by 

(4.3) L = J\ + j2C'j{J-j + iy. + J. 

Moreover, J of these pieces are (J — 1) -dimensional linear facets along the coordinate axes 
while the remaining B = L — J ones are in the interior of R^ and form 0{i), which are 
linear or smooth curved facets hk{c,i) on R'^ for k ^ U = {1,2, ...,B}, i.e., 

(4.4) 7^(^) = \ceRi: hk{c, i)<0, keu]. 

Moreover, if CMAc{P-iH{i)) is used to denote the sum capacity upper hound for the MAC 
capacity region, then 

J 

(4-5) hk^^^ (c, i) = Y, Ci - Cmac{P, H{i)) 
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where kMAC is the index corresponding to Cmac{P, H{i)). 



Example 4.1 For the MAC channel and each i E fC, when J = 2 and N = 1 (i.e., each of 
the user's mobiles has only single transmit antenna), it follows from [25] that 

51 (c, i) = ci - log I / + Hi (i)PiHi (i) I , 
52(c,i) = ci + C2 - log |l + Hlii)PiHi{i) + Hl{i)P2H2{{)\ , 
93{c,i) =C2- log |/ + (/ + Hl{i)PiHi{i))-^Hl{i)P2H2{i) 

4.2 The MIMO BC Capacity Region 

In the MIMO BC and for each channel state i e K., let X{i) G C^^^ denote the transmitted 
vector signal from the base station and let Yj{i) G (j^xi i^g the received signal at the user j. 
The noise at user j is represented by Nj € C^^^ and is assumed to be circularly symmetric 
complex Gaussian noise {Nj ~ N(0,I)). The received signal of user j (see, e.g., Figure 3) is 
equal to 



Yj{i)=Hj{i)X{i)+Nj. 



The base 



The transmit covariance matrix of the input signal is Tx{i) = E X{i)X^{i) 
station is subject to an average power constraint, which implies that Tr(rx(i)) < P- During 

each channel state i £ IC, the J-user MIMO BC capacity region denoted by TZ{i) can be 
calculated by the duality of the MAC and the BC in [31] and [25], where the BC capacity 
region is obtained by taking the convex hull of the union over the set of capacity regions of 
the dual MIMO MACs such that the total MAC power is the same as the power in the BC, 



I.e., 



(4.6) 



n{i^=CBc{P,H{i)) 



u 

m,...,Pjy.J2'Pj=p} 



Cmac{Pi,...,Pj,H\i)). 



Moreover, the Dirty Paper Coding (DPC) proposed in [15] achieves the capacity for the 
MIMO BC (see, e.g., [50]). In particular, if each user has only single receive antenna, we 
have the following lemma. 

Lemma 4.2 For the J-user MIMO BC with N = 1, each i E K, and L given in (4.3), TZ{i) 
contains the origin and has L boundary pieces of which J are (( J—l)- dimensional linear facets 
along the coordinate axes while the remaining B = L — J ones are in the interior of and 
form 0{i), which are linear or smooth curved facets hh{c, i) on R-^ for k eU = {1,2, B}, 
i.e.. 



(4.7) 



7^(^) = {ceR'l: hk{c, i) < 0, G w} . 
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Moreover, if Csato{P, H{i)) denotes the sum capacity upper bound (called the Sato upper 
bound) for the BC capacity region, then 



J 



(4.8) 



hksatoic,i) = ^Cj- Csato{P,H{i)) 



where ksato &U is the index corresponding to C Sato{Pi H (i)) . 

Remark 4.1 The concept o/Sato upper bound of the J -user MIMO BC can be found in such 
as [25], [9], [4-^], [45] 7 [55]. Moreover, here we conjecture that, in the MIMO BC channel, 
if each user has multiple receive antennas, the corresponding property stated in the lemma 
should also be true. 

Example 4.2 Considering the BC channel with J = 2 and N = 1 for each i £ IC, we can 
derive hk{c,i) explicitly for fc = 1,2,3 by employing the results in [25], [46], [50] as follows 



5 Proof of Theorem 3.1 

To be convenient for readers, we first outline the proof of Theorem 3.1, which consists of the 
following five parts. 

Firstly, in Subsection 5.1, we first justify a dual relationship between the utility-maximization 
problem in (2.5) and the cost-minimization problem in (2.12), which is summarized in 
Lemma 5.2. Then we prove a claim in Lemma 5.3, which states that when the system 
state is close to the unique optimal solution to the cost minimization problem (called a fixed 
point), the capacity of the system will be fully utilized. The claims stated in Lemmas 5.2-5.3 
are similar to their counterparts in [54], nevertheless, their concrete proofs are different due 
to the different problem formulations and the difference of the capacity constraints between 
the two studies. 

Secondly, in Subsection 5.2, we present an equivalent queueing model due to the as- 
sumption (3.2) imposed on the FS-CTMC and justify a functional central limit theorem 
(Lemma 5.4) for a DSRP whose arrival rate process is driven by the FS-CTMC. The main 
idea used in proving Lemma 5.4 is stemmed from the related discussion in [18], [17] and the 
concrete proving techniques include the conventional functional central limit theorem (see, 
e.g., [28] and [38]), random change of time lemma (see, e.g., [5]), establishment of oscillation 



(4.9) 



hi{c,i) 



e2(ci+c2) _ I J + (Hl{i)H,{i) - Hl{i)H2{i)) 



(4.10) 
(4.11) 



/12 (c,i) 
h^{c,i) 
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inequality (see, e.g., [18], [17]), equivalent conditions of relative compactness and Skorohod 
representation theorem (see, e.g., [23]), and etc. 

Thirdly, in Subsection 5.3, we derive the fluid limit processes for the physical processes 
under fluid scaling in Lemma 5.5 and study the asymptotic behavior for the fluid limit 
processes as time evolves in Lemma 5.6. Fluid limits arc widely used as an intermediate step 
in justifying diffusion approximations (see, e.g., [7], [43], [54], [19], [3], [4], and references 
therein). Nevertheless, our fluid limit is a random process driven by the FS-CTMC rather 
than a deterministic function of time as obtained in the existing studies. This new feature 
brings us additional complexity in proving Lemma 5.5 and Lemma 5.6, e.g., comparing with 
the study in [54], it requires more technical treatment in handling the FS-CTMC based 
jumps for the constructed Lyapunov function. Therefore, by noticing this new feature and 
the difference between our optimal scheduling policy and the one in [54], we develop a theory 
through combining and generalizing the discussions in [54], [16], [3], and [4] to finish the 
justifications of Lemma 5.5 and Lemma 5.6. 

Fourthly, in Subsection 5.4, we study the convergence of the workload and queue length 
processes on a finer time-scale, which is an important step in justifying the main result of 
the paper. This method has appeared in queueing literature for a while (see, e.g., [6], [54], 
[43], [36], [41], and etc.) The main difference between ours and the existing works is as 
follows: all the processes concerned in our study involve the jumps introduced by the random 
environment and in the meanwhile the processes in existing studies do not involve this type 
of jumps. Therefore we develop a scheme and incorporate it into the framework as used in 
[54] to finish the proof of the convergence properties for the processes on a finer time-scale. 

Finally, in Subsection 5.5, we combine the results obtained in the previous subsections 
with the uniqueness of solution to an associated Skorohod problem and the minimality of the 
Skorohod problem to provide a proof for Theorem 3.1. This type of techniques have been 
used in the studies concerning network scheduling (see, e.g., [54], [43], [36], [41], and etc.) 
Nevertheless, our justification logic and technical treatment are somewhat different. 

5.1 Preliminary Lemmas on the Utility- Msiximization and Dual Cost Minimiza- 
tion Problems 

Lemma 5.1 Consider the utility-maximization problem in (2.5) and suppose conditions (2.6)- 
(2.9) and (2.11) are imposed, then for a sequence of queue states, {q\l = 1,2,...}, which 
satisfies q'' q E R'^. as I oc, we have 

(5.1) Kj{q\i) ^ kj{q,i) as I ^ oo 

for each i G IC and any j G J such that qj > 0. 

Proof. Consider each specific state i G K., then the proof can be accomplished similarly as 
for Lemma 6.2 in [53] and hence we omit it. □ 

Lemma 5.2 For each state i G fC, the following claims are true. 
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1. Under the policy in (2.10) and the associated convention, if c* = p{i) is an optimal 
solution to the maximization problem in (2.5) with a queue state q in the utility function. 
Then q* = q must be the optimal solution to the minimization problem in (2.12) with 



c = c* in the cost function and with w = J2j=i Qj/l^j the constraints. 



2. Conversely, if q* is the optimal solution to the minimization problem in (2.12) with 
w > Q and A{q*,i) = p{i) for each i E JC in the cost function. Then q* > and 
A*{q*,i) = A(g*,z) must be an optimal solution to the maximization problem in (2.5) 
with q = q* in the utility function. 

Proof. First of all, without loss of generality, we suppose that g > 0. Then it follows from 
the KKT optimality conditions (see, e.g., [35]) that the solution to the utility maximization 
problem in (2.5) can be obtained through the following equations, 

(5.3) rikhk{c,i)=0 for each keU 

where B and U are defined in (2.2), r]k > for all A; € W are the Lagrangian multipliers 
and hk{c,i) for each k & U and i G /C is defined in (2.2). Similarly, the solution to the cost 
minimization problem (2.12) can be obtained through the following equations, 

(5.4) qj [ ^^'^^^'""'^ + — ) = for each j G J, 
(5.5) 

where > is the Lagrangian multiplier. Moreover, it follows from (2.14) that 
(5.6) 




dCj{qj,Cj) _ 1 dUj{qj,Cj] 



dqj iXj dcj 

Thus, based on the above facts, the claim in the first part of the lemma can be proved 
as follows. By condition (2.7), we know that J2j=i Uj{qj,Cj) is strictly concave in c for each 
q> 0. Therefore c* = p{i) is the unique optimal solution to the utility maximization problem 
in (2.5) for the given g > in the utility function, which satisfies (5.2)-(5.3). Thus, if we take 

^ dhk{p{i),i) 



then it follows from (5.2) and (5.6) that (5.4) holds. Due to condition (2.9), wc know that 
V{q, c) is strictly convex in q for each c > 0. So the cost minimization problem in (2.12) has a 
unique optimal solution q* = q when c = c* = p{i) is in the cost function and w = J2 j=i Qj I l^j 
is in the constraints. 
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Conversely, the claim in the second part of the lemma can be proved as follows. Due 
to the conditions (2.8)-(2.9) and the relationship (2.14), we know that V{q,p{i)) is strictly 
convex in q. Therefore q* is the unique optimal solution to the cost minimization problem 
(2.12) with A{q*,i) = p{i). Thus we can prove q* > Ohj showing a contradiction. 

In fact, without loss of generality, we suppose that there is some m ^ J with m < J such 
that q* € Q{ki, ...,km) with ki ^ 1 and k„i = J, where Q(ki, ...,km) is defined in (2.10). 
Then we can construct a 2-dimensional line for some constant e > w, 

(5.7) P,: ^ + ^+ V ^ = e>w 

such that it passes through the point q*. Now it follows from (2.14) that the function 
f{qi,p{i)) (= V{q,p{i))) with the constraint Pi for all q = (gi, •••i 9j-n ^ ^+) of 
the following derivative function in qi G 7^]*", 

.gg^ df{qi,p(i)) ^ 1 dUi{qi,pi{i)) 1 dUj{{e - - J:m,jjeJ li)f^J^ pA^)) 
dqi Hi dci Hi dcj 

which is strictly increasing in qi G Tli due to (2.8). Moreover, it follows from (5.8) and (2.9) 
that 

0/(0, p(z)) _ 1 dUj{{e - j:j^i,j,jej ^ ^ 

lo.yj — 5 — 5 < u, 

oqi m dcj 

(5.10) dfilhPm ^ 1 dUiiql,pi{i)) ^ ^ 

dqi pi dci 

Then, by (5.9) and (5.10), we know that there is a gi G {0,qi) such that 

(5.11) dm,p{i)) ^ 

dqi 

which implies that, on the curve f{q,p{i)) with q = (gi,(/|, ■■■,q*j-i,qj)' € there exists a 
minimal point q G TZ\. with q = {qi,q2, ■■■iq*j-iiqj)' such that y{qiP{i)) < V{q* ,p{i)) 

~ ( ^ 

9J = e 2^ — \ PJ- 

This contradicts the assumption that q* is the optimal solution to the cost minimization 
problem in (2.12). Hence we can conclude that q* > 0. 

Finally, if q* is the optimal solution to (2.12) with c = A{q*,i) = p{i) in the cost function, 
we see that (5.4)-(5.5) hold with q = q* and c = A{q* ,i) = p{i). Therefore we can take r/^^ = 9 
and r]k = when k ^ ku in (5.2)-(5.3) since g* > and p is on the curve hk^{c, i) = 0. Hence 
A*{q*,i) = A{q*,i) = p{i) for each z G /C is an optimal solution to (2.5) with q = q* \n the 
utility function. □ 

Next, let II • II denote the norm of a vector g G i?"^ in the sense that ||g|| = Z]/=i \qj\- Then 
we have the following lemma. 
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Lemma 5.3 For each state i & JC, the following claims are true. 



1. The cost minimization problem (2.12) has a unique optimal solution q* = q when c = 
p{i) is in the cost function, and moreover, q*{w,p{i)) is continuous in terms ofw. 

2. Assuming that, for any given constant e > 0, there exists another constant a > 0, which 
depends only on e, such that, for any q G V(e, a, i) with 

(5.12) V{€,a,i) = Iq e R'l : \\q- q*{w,p{i))\\ < a and w = '^—qj >e>, 
I .=1 J 



we have 
(5.13) 



J J 
^Aj{q,i) = J2pj{i) 

3=1 j=l 



Remark 5.1 The unique optimal solution q*{w,p{i)) will be referred to as a fixed point in 
the following discussion. 

Proof. For the first part in the lemma, we have the following observations. Due to 
condition (2.9), we know that V{q,c) is strictly convex in q for each c > 0. So the cost 
minimization problem in (2.12) has a unique optimal solution q* = q when c = p{i) is in the 
cost function. Moreover, the continuity of q*{w,p{i)) in terms of w for each i G /C can be 
proved similarly as in [54]. 

For the second part in the lemma, it can be proved by showing a contradiction. As a 
matter of fact, if the claim is not true for some i E K. and some e > 0, then for a sequence 
of a' I along I G {1,2,...}, there is a sequence of states G V{e,a'',i) with I = 1,2,... 
satisfying 



(5.14) 
(5.15) 

such that 
(5.16) 



q'' - q*{w\p{i)) -)■ as Z -)■ oo. 



w 



Y —q\ >e for all Z = l,2,... 

H Mi 



J J 
Y,Aj{q\i)<Y.Pj(i) for ah Z = l,2,... 

3=1 3=1 



Otherwise, if there is some Iq G {1,2,...} such that V{e,a\i) are empty for all I > Iq, then 
(5.13) is automatically true for the given i e JC and e > 0, which is a contradiction. Now, let 



(5.17) 



qi = {e/w^)q^ so that u;' = ^ ^ = e for all 1 = 1, 2, 

3=1 
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then it follows from (5.14)-(5.15), (5.17), (2.11) and Lemma 5.2 that, along / G {1,2, ...}, 
(5.18) = + 4^ (ql _ q*iw\pi{))) ^ q = q*{e,p{z)) > 0, 

which implies that > for all large enough I G {1, 2, ...}, and moreover, we also have 

eq*{w\p{i)) . 



(5.19) 



Hence, by (5.18), (2.7), (2.10), and the similar proof as used for the second part of Lemma 5.2, 
we have, for all large enough / G {1, 2, ...}, 

(5.20) A(^,i)>0 and A(g',i)>0. 
Furthermore, by (2.11) and (5.16), we have, for each / G {1,2,...}, 

(5.21) X^A,(g',z)<Ep.«- 

Thus it follows from (5.21) and Lemma 5.1 that 

J J 

(5.22) EA,(g,i)<Ep,(i). 

i=i i=i 

Notice that the condition in (2.7) and the fact in (5.18) imply that A(g,i) and K{q\i) for 
large enough / can only locate on the capacity surface of 7?.(i) (that is defined in (2.2)). Then, 
by combining this fact with (5.21)-(5.22) and Lemma 5.1, we can see that A(g, i) can not be 
in the interior of the facet corresponding to Cu- Hence we can conclude that there is some 
j G J, e.g., without loss of generality, take j = 1 such that 

(5.23) ki{q,i) < pi{i). 

So, on the one hand, it follows from (5.20) and (5.2) that there exists a set of Lagrange 
multipliers {qjk > 0, fc G {1, B + J}, j G J^} such that 

ft^OA\ dhkiA{q,i),i) ^ dUj{qj,Aj{q,i)) 

j=ik=i j=i 

dUj{qj,pj{i)) 
^ dcj 

where B is defined in (2.2), the first equality of (5.24) follows from (5.2)-(5.3) and (5.20), 
the second equality follows from (5.21), (2.11) and (5.1), and the third equality follows from 
Lemma 5.2. 
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On the other hand, due to the strict concavity of Uj{qj,Cj) in cj for each j E as stated 
in (2.7), it follows from (5.20)-(5.23) that 

(5.25) £j^^^ ^h,iA{q,^),^) 
j=i k=i '^^i 
^ dUi{qi,pi{i)) _ ^.^ dUj{qj,Aj{q*{w^,p{i)),i)) 
dci i^oo .Jr^^^ dcj 

dUj{qj,pj{i)) 

^ dcj 

j=i J 

Obviously, there is a contradiction between (5.24) and (5.25). Thus the assumption stated 
in (5.14)-(5.16) is not true, which implies that the second claim in the lemma holds for the 
third case. Hence we finish the proof of Lemma 5.3. □ 



5.2 Equivalent Processes in Distribution 

First of all, we define a sequence of jump times in terms of the FS-CTMC process a(-) as 
follows, 

(5.26) ro = 0, Tn = mf{t>Tn-i:a{t)^a{t-)}. 

So it follows from Proposition 5.2.1 in page 376 of [39] that r„ — )■ oo a.s. as n — >■ oo and 
each sample path of «(•) has at most finitely many jump points over any bounded interval 

[0,T] since the state space of a(-) is finite. Moreover, we introduce a new stochastic process 
a.(-) induced by a(-) up to each given time t G [0, oo), i.e., at{s) = a{s) with s G [0,t\. In 
addition, for each j G {1, J}, let 

(5.27) A'j{r^;a.{-)) ^ {A^j{rH,at{-)),t G [0,oo)} 

denote the counting process that the arrival rate corresponding to A^(r^-, a.(-)) during time 
interval [r^r„, r^r„-|-i) is A^(a(r„)) for n G {0,1,2,...,}. Similarly, let 

(5.28) A^^{r^.,a;,X-)) ^ {A^^{rH,al,,{-)),t G [0,oo)} 

denote the corresponding process with arrival rate A^(a'"(r^)) during time interval [t^,t^_|_i) 
for each n G {0, 1, 2, }, where {T^,n G {0, 1, }} is a sequence of jump times in terms of 
«''(■). Due to the second condition in (3.2), the definition of DSRP, the assumptions among 
the arrival and FS-CTMC fading processes, and Theorem 5.4 in page 85 of [32], we have, 

(5.29) E^-)^A^j{r'.,a.{-))=^A^^{r'. 

where the notation denotes "equals in distribution". Then it follows from (2.15) and the 
assumptions among the arrival, service and FS-CTMC fading processes again that 

(5.30) Q^j{.) Im-) - ls]{T^,{-)) 
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where 

(5.31) f;{.) ^ Aj (Q^is), a{s)) ds ^T^ir'-), 

(5.32) OK*) ^ ^Qjir't) 

and we have used the radial homogeneity of A{q,i) in (2.11) for (5.31). Now, let 

(5.33) E^{-) = {El{-),...,E}{-)y with E^{-) = i (^^r^-, «(•))- r^A^-, «.(•))) , 

(5.34) S^{.) = {Sl{.),...,S^j{.)y with S;(-) = i ^Sj{r'.)-njr'.) 
for each j G {1, J} with 

(5.35) A^-, «.(•)) = X'j{a{s))ds 

j^\]{a'{r'^s))ds 

= \ f yja''{s))ds. 
Jo 

Moreover, define 

(5.36) = (Ai(-, a.(-)), ...,>}(•, a.(-)))' . 
Then we have the following lemma. 

Lemma 5.4 For the diffusion-scaled processes in (5.33)-(5.34), the following convergence in 
distribution is true as r ^ oo, i.e., 

(5.37) (e^(-), S'{-)) ^ (i?^(-), iry/^B^i-)) . 
where = diagd^iPf, 

Proof. It follows from the heavy traffic condition (3.2), the functional central limit theorem 
(see, e.g., [28] and [38]), and the random change of time lemma (see, e.g., page 151 of [5]), 
Lemma 8.4 in [17] that, for each n G {0, 1, ...}, 

(5.38) [E^Tn + t)- p-{Tn)) l{0<t<a„} 

= ^ [A'-{r\Tn + t),ar„+ti-)) - A'{r\n,arA-))) l{0<t<a„} 

-r (Y(r„ + t,ar„+t(-)) - A''(r„, ar„(-))) Ao<t«T„} 
= ^i' (r2(t - </>n/r2)/{o<t<^„}) + en - r (A^(r„ + t, a^„+t(-)) - A^(rn, arJO)) ^{o<t<a„} 

^ (r^V(r„))) ^ /{o<t<.„}S^(t) as r ^ cx) 

(if^(r„ + t)-i7^(r„))7{o<t<a„}' 
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where (T„ = r„+i — t„ is an exponentially distributed random variable independent of all 
other random events concerned since a(-) is a FS-CTMC, e„ = (e^, e;J)' with e{ = if 
Ej{-) has a jump at t„ and 1 otherwise for each j € {1, J}, is a renewal process with 

rate vector X^{a{Tn)) = (Ai(a(r„)), Aj(Q;(r„)))' and 

with = (0^, !?!>;()' being a J-dimensional random vector whose jth component ^ for 
each j G {1, J} denotes the remaining arrival time beginning at r„ for a packet to the _7th 
queue with rate Aj(a(T„)) switched from Aj(a(r„_i)) at t„ for each n G {1,2, ...}. Moreover, 
to be convenient for later purpose, we reexpress (5.38) as follows, over each [r„,rn,+i) and as 
r — >■ oo, 

(5.39) E^'^i-) = E^{rn + -)-E'{Tn) 

Then, by following (5.39) and by generalizing the discussion in the proof for Theorem 3.2 in 
[18] or Lemma 8.2 in [17], we can reach a proof for the claim in (5.37). 

To do so, we first establish the relative compactness for E^{-) with r G {1, 2, }. As a 
matter of fact, define the modulus of continuity in terms of a function x{-) : [0, oo) — ^ 
with some integer d > for each given T > and (5 > as follows, 

(5.40) w{x,S,T) = inf maxOsc(a;, [t/_i,i/)) 

tl I 

where the infimum takes over the finite sets {ti} of points satisfying = to < ti < ... < tm = T 
and tl — ti-i > 5 for / = 1, m, and 

(5.41) Osc{x,[ti^i,ti]) = sup \\x{t)-x{s)\\2 

ti<S<t<t2 

with II • II2 denoting the Euclidean norm in R'^. Then it follows from Corollary 7.4 in page 129 
of [23] that the justification of the relative compactness is equivalent to proving the following 
two conditions: 

(a) For each 77 > and rational t>0, there exists a constant c{r], t) such that 

liininf p{||E''(i) ^ < c(ry,t)} > 1 - 77. 

(b) For each rj > and T > 0, there exists a S > such that 

limsupP < w{E^ ,6, T) > rj} < rj. 

To show (a), we first define N{t) = max{n, Tn < t} for each t G (0, 00). Then, for 
each rational t > 0, take a T > such that t G (0, T] and define a sequence of events: 
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Si = {oj : N{T,uj) < 1} for each I € {1, 2, }. Since a(-) has at most finitely many jumps 
a.s. over [0, T], we know that the sequence of probabihties P{Si} increases monotonously to 
the unity as Z — >■ oo. Thus, for the given rj > 0, there is some large enough L > such that 



(5.42) 



P{Sl} > 1 - f . 



Moreover, it follows from (5.39) and Remark 7.3 in page 129 of [23] that E^{-) satisfies the 
following compact containment condition, i.e., for each r/ > and T > 0, there is a constant 
Kn > for each n G {0, 1, } such that 

(5.43)inf PjT'"} > 1 - 7^ with T'" = {u : \\E'{t)\\2 < Kn,t e [0,T] D [0,a„)}. 

In addition, for each n G {1, 2, }, let A„ = ((5^, 6^)' with (5^ = 1 if -EJ(-) has a jump at 
Tn and zero otherwise for each j G {1, J}. Then, for each t G [Tiv(t),Tjv(t)+i), we have 



(5.44) 
(5.45) 



^'-(t) = ^'-(rivw) + ^'''^(*Hi-T^(t)), 



Therefore it follows from (5.44)-(5.45) that, along each sample path and for any ti,t2 G [0, T], 

(5.4S^C [h,t2]) < E OSC (E"^'", [h - TnM - Tn] H [0, (7„)) + -{N{t2) - N{h)). 



n=0 



Thus it follows from (5.46) that, along each sample path in <Sl fl 7^'" with r, n G {1, 2, ...}, 



(5.47) 



E^it)\l < \\&-{0)\l + Osc[E^,[0,t]) 



< 2V sup £;^'"(t) 
n=o«e[o.^]n[o,o-„ 



L 

+ -. 

2 r 



Hence, for the above arbitrarily given r) > 0, each rational t G [0, T], and large enough 
r G {1, 2, ...}, we know that 

(5.48) p|||E^(t)||^<2^+i|^^K„| 

> P{Sl} - E ^{{||^''"W|L > (2^" - ;:^) } fl'^L for some t G [0,T] n [0,a„)| 



n=0 

L 



> P{5l} - E^{{||^''"(0||2 >^n}n'5L for sometG [0,T]n[0,a„)} 



n=0 
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where the second inequahty follows from (5.47) and the fact that 

P{\\aX + bY\\2>K^ + K2}<p!^\\X\\2 > ^} + ^{li^ll2 > ^} 

for any real number a, b and random vectors X, Y. Moreover, the last inequality in (5.48) 
follows from (5.42) and (5.43). Thus condition (a) holds. 

Next we prove the condition (b) to be true. Due to (5.39), we know that, for each rj > 
and T > 0, there exists a Sn > for each n € {0, 1, } such that 

(5.49) hmsup P |ty(E'-'", 6n, [0, T] n [0, a„)) > ^| < 

Now take S = min{So, 6l} > 0, then for each r G {1, 2, } and each sample path in Sl, 

J T 

(5.50) w{E\5,T) < ^«;(^'-'-,(5,[0,r]n[0,a„)) + - 

n=0 ^ 

■r T 
< ^w;(^^'",<5„,[0,T]n[0,a„)) + 



r 

n=0 



where the first inequality follows from (5.40) and (5.46), and the second inequality follows 
from 1.9 in page 326 of [29]. Therefore, for each large enough r G {1, 2, }, it follows from 
(5.49)-(5.50) that 

p{w{E'',5,T) > 77} 

< |+p{{^^^5,^)>r?}^5L} 

< 1 + |:^'{{«^(^'■'^5„,[o,^] n [o,an)) > ^ (i - J)}n5L} 

< f + E ^ { {«;(^'■'^ Sn, [0, T] n [0, an)) > ^} fl '^l} 

< ri. 

So the condition (b) is true and hence we know that E'^{-) is relatively compact for r G 
{1,2,...,}. 

Finally, consider any subsequence TZi C {1,2,...,} such that, along r G T^i, we have 
(5.51) E^{-) ^ E{-) (a process to be identified). 

Then it follows from the Skorohod representation theorem (see, e.g.. Theorem 3.1.8 in page 
102 of [23]) and the random change of time lemma (see, e.g., page 151 of [5]) that, for each 
n G {0, 1, } and along r G T^i, 

(^^(•)^{.<r„+,},^^(-)^{.<r„}) ^ (^(•)/{.<r„+,}'^(-)^{-<r„}) • 
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Then, by the method of induction in terms of n G {0,1,...,}, (5.39), and the continuous- 
mapping theorem (see, e.g.. Theorem 3.4.1 in page 85 of [52]), we can conclude that, along 
r G TZi, the limit in (5.51) is H^{-). Moreover, since TZi is arbitrarily chosen, we know 
that ^ along r G {1, 2, }. Moreover, by the independence assumptions and the 

functional central limit theorem, we know that the claim in Lemma 5.4 is true. □ 



5.3 Fluid Limiting Processes 

For each j G JT", t > and r > 0, we define the fluid-scaled processes as follows. 



(5.52) W'it) ^ y,W^{rh), Y^{t) = ^.Y^irH), ^{t) ^ l^;(t), S^^{t) ^ ^S^^{rH) 



and use Q'^ {■) , {■) , S'^ {■) , T^{-) to denote the corresponding vector processes. Further, let 

(5.53) Qjit) = Qj{0) + Xjit) - lijfj{t) for each j G J, 

(5.54) W{t) = = W{Q) + y(t), 

(5.55) Y{t) = (^J^ pj{ais))ds - r,(t)) , 

(5.56) Tj{t)= f\j{Q{s),a{s))ds, 

Jo 



where, for each i G /C, 
(5.57) Hq,i) = i 
and, for each j & J', 



\Q{ki,-,km)(^q^j_y if q e Q{ki, ...,km) and a given me J, 
p{i) if q = 0. 



(5.58) A.^^''''-'''"'\q,i) = I ^^'^^'^^ = Af^'"'""'''"'\q,i) if j ki,l E {l,...,m}, 

^ ' \ Pj{i) a j = ki,l e {l,...,m}. 

Thus we have the following lemma. 

Lemma 5.5 Suppose Q^'{0) => Q{0) as r ^ oo, then under the utility-maximization alloca- 
tion policy A{q,i) in (2.10), any subsequence of {r,r = 1,2,...,} has a further subsequence 
{ri,l = 1,2, ...} such that the following convergence in distribution is true, 

(5.59) {E^'i-),S^'i-),f'^{-),Q'^i-),W''i-),Y^^i-))^{E^^^ 

as I —> oo, where the limit in (5.59) satisfies (5.53)-(5.57). Moreover, if Q{0) = 0, then the 
convergence in (5.59) is true along the whole sequence r = 1,2, ... with the limit satisfying 

(5.60) E{t) = ~X{t,at{-)), S{t) = p{t), r(t) = c(t, «*(■)), 

(5.61) Qj{t) = W{t) = Y{t) = 0, 
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for each t >0 and j G J7, where jx{t) = (/xi, fXjYt, and 

(5.62) X{t,at{-)) = {Mt,at{-)),...,h{t,at{-)))\ Xj{t,at{-))= f\j{ais))ds, 

Jo 

(5.63) c{t,at{-)) = {ci{t,at{-)),...,cj{t,at{-))y , Cj{t,at{-))= pj{a{s))ds. 

Jo 

Proof. It follows from (5.31) and (2.16) that T"'(-) is a.s. uniformly Lipschitz contin- 
uous with Lipschitz constant maxjgx;(Z^/=i Pj(0) ^ach r > 0, which implies that it is 
absolutely continuous and differentiable at almost every t G (0, oo) (in other words, almost 
every t G (0, oo) is a regular point of T'^(-)). Thus the sequence of stochastic processes 
{T^{-),r = 1,2,...} is C-tight, that is, it is tight and each weak limit point is in C[0,oo)'^ 
a.s., where C[0, oo)'^ is the space of all J-dimensional continuous functions over [0, oo) and 
is endowed with the Skorohod Ji-topology (see, e.g. Page 116 of [23]). Moreover, it follows 
from Lemma 5.4 that (E"- {■) , S'' {■)) is also C-tight. In addition, by (5.30)-(5.31) and (5.52), 
we know that 

(5.64) Q^^{t) = E^it)-S^if;{t)). 

Hence it follows from (5.64), (2.17) and the random time change lemma in page 151 of [5] 
that the following sequence is C-tight as well, 

(5.65) {E'i-),S'i-),f'i-),Q^i-),W'i-),Y'i-)) , 

where we have used the independent assumption related to {a'^{-),r = 1,2,...}, the second 
condition in (3.2) and the fact that 

(5.66) \ r pj{a'{s))ds / pAa{s))ds. 

r^ Jo Jo 

Therefore, any subsequence of the processes in (5.65) has a further subsequence convergent 
in distribution. Now suppose {E{-), S{-),T{-),Q{-),W{-),Y{-)) is a weak limit point corre- 
sponding to the further subsequence indexed by {ri,l = 1,2,...}. Then, by the Skorohod 
representation theorem (see, e.g.. Theorem 3.1.8 in page 102 of [23]), there is a common 
supporting probability space such that 

(5.67) {E^^{.),S^^{.),T^'{.),Q^'{.),W^'{.),Y'''{-)) ^ (A(.,a.(-)),M-),r(-),Q(-),t^(-),^'(-)) 

u.o.c. a.s. as Z — >■ oo and the limiting processes in (5.67) satisfy (5.53)-(5.57). Here we only 
need to justify (5.57) to be true and other equations hold obviously. 

As a matter of fact, due to (5.67), we know that the limit processes in (5.67) arc uniformly 
Lipschitz continuous a.s. So our discussion will base on a fix sample path and each regular 
point t > over an interval (r„_i,r„) with n G {1,2, ...} for Tj with j G JT". It follows from 
(5.53) that Q is differential at t and satisfies 

dQiit) , , , dfJt) 

(5.68) J^ = ,.(„(t„_^.^ 
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for each j G J. If Qj{t) = for some j G J, then it fohows from Qj{-) > that 

(5.69) ' = which imphes that = ^ = pj{a{t)). 

If Qj{t) > for the j G JT", then there exists a finite interval (o, 6) G [0, oo) containing t in it 
such that Qj{s) > for all s G {a,b) and hence we can take small enough S > such that 
Qj{t + s) > with s G (0,6). Thus it follows from (5.31) and (5.1) that 



(5.70) 



^^{f;-{t + S)-fJ''{t))-A,{Q{t),a{t)) 

< \ \Ai{Q''''{t + s),a{t + s))-Aj{Q{t + s),a{t + s))\ds 
Jo 

+\ [ \Aj{Q{t + s),a{t + s)) - AiiQ{t),ait))\ds 
Jo 

- / \Aj{Q{t + s),a{t + s)) - Aj{Q{t),a{t))\ds as k ^ 
Jo 



oo 



where wc have used the Lcbcsguc dominated convergence theorem for the last claim in (5.70). 
Due to the right-continuity of «(•), the Lipschitz continuity of Q{-) and (5.1), the last ex- 
pression in (5.70) tends to zero as S 0"*". Hence we have 



(5.71) ^ = = A,(Q(t),a(t)) for each j G J 



which implies that the claims in (5.56)-(5.57) are true. 

Next, we introduce the following cost objective with c{i) = p{i) in (2.14) for each i & K,, 

J 

(5.72) ^l;iq,i)^Y.Cj{qj,Pj{i))- 

Then, for each regular time t > oi Q{t) over time interval (r„_i,T„) with a given n G 
{1, 2, ...}, we have 

. MQ{t),a{t)) ^ '( dQ,{t)dCj{Qj{t),p,{a{t)) dpj{a{t)) dC.jQ.jt), p,{a{t)) \ 
^ ' ' dt fr{\ dt dQj{t) dt dpj{a{t)) J 

= E («(«(*» - A,{Q(t). «(»))) ''^"|^g^;'"" %..)>., 

< 

where we have used (5.68), (5.71), (2.14) and the fact that the sample paths of «(•) are 
piecewise constants for the second equality of (5.73), and we have used the concavity of 
the utility functions, the fact that Aj{Q{t),a{t)) is the optimal solution to (2.5), and the 
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similar arguments as used in [54] for the last inequality of (5.73). Therefore, for any given 
n G {0, 1,2, ...} and each t € [r„,r„_)_i), we have, 

(5.74) < i;{Q{t),a{t)) 

< i'{Q{Tn),Oi{Tn)) 

< KV'(Q(0),a(0)), 

where the second inequality in (5.74) follows from (5.73) and the last equality in (5.74) follows 
from (2.14), (2.7)-(2.8), (2.18), the continuity of Q{t) at all jump times r„ with n G {1, 2, }. 
Moreover, the n in the last inequality of (5.74) is a positive constant given by 



m- 



whcre the indices in each of the M products of (5.75) satisfy that ik ^ ii k ^ I for 
k, I G {0, 1, m} and the integer M can be explicitly calculated since the state space of a(-) 
is finite. 

If 0(0) = 0, it follows from (5.74) that Q{t) = for all t > 0. Thus, by (5.54), we know 

that W{t) = Y{t) = for all t > 0. Moreover, it follows from(5.69) that the third claim 
in (5.60) is true. Hence, under the assumption that Q{0) = 0, all the claims stated in the 
lemma arc true. □ 

Next, since yj = Cj(qj, pj{i)) is strictly increasing in qj for each i € JC and j G J', its 
inverse C~^{yj, pj{i)) is well defined and is strictly increasing in yj. So, for each k > 0, we 
can define 

K K J 

(5.76) 5(k) = ^ max 'i/'(g,i) and g{K) = Y,Y.'^i'^(9{i^), Pjii))- 
Then we have the following lemma. 

Lemma 5.6 Under the same conditions as used in Lemma 5.5, if ||(5(0)|| < % for some 
constant x, then Q{t) is bounded for each t >Q, i.e., 

(5.77) \\Q{t)\\<9{x) for each t>0. 
Moreover, there exists a time T^^e > for any given e > such that 

(5.78) \\Q{t) - q*{W{t),p{am\\ < e for all t > T^,^ 

and in particularly, if Q{0) = q* {W (0) , p{a{0))) , then Q{t) = Q{0) a.s. for all t G [to,ti). 
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Proof. If ||0(0)|| < Xi it follows from (5.74) that Q{t) is bounded for all t > since 
Cj{qj, pj{i)) for each j ^ J and i € /C is strictly increasing and unbounded function in qj. 
Moreover, it follows from (5.76) that g{K) is increasing in n with ^(O) = and hence it follows 
from (5.74) and (5.72) that 

(5.79) C,{Qj{t),p,{am<~9{x) and Qj{t)<Cj\g{x),pMt))) 

for each t > and j G J, which implies that (5.77) is true and W{t) increases to some finite 
number as t increases due to (5.54)- (5.55) and (5.77), i.e., 

(5.80) W{t) t W{oo) <oo as t ^ oo. 

Thus we can define the following Lyapunov function with at most countably many jumps, 

(5.81) L{Q{t),a{t))=2l;{Q{t),a{t))-2l;{q*{W{t),p{am,o^{t)), 

which is nonnegative and bounded over t G [0, oo) due to Lemma 5.3, (5.54), Lemma 5.5, 
(5.80) and the fact that Q{t) and p{a(t)) are bounded over [0,oo). Then, for any given 
regular time t > over an interval (r„_i,r„) with n G {1, 2, ...} and for any S > 0, we can 
show that there exists a a > such that 

dLiQ{t)^,a{t)) ^ _^ \\Q(t)-q*{W{t),p{am\\ > S. 

As a matter of fact, it follows from (5.73) and (5.80) that ip{Q{t),a{t)) is non-increasing 
and ijj{q*{W{t), p{a{t))), a{t)) is non-decreasing in t G (r„_i, r„) since a{t) keeps fiat over the 
time interval (rn-i,rn). So we only need to show (5.82) true with respect to ijj{Q{t),a{t)). 
By (5.73), we define 

(5.83) hm),ait)) ^ 



dt 

J 



dU,{Qj{t),pj{a{t))) 



which is continuous in terms of Q{t) = q ^ with q G due to (5.1), (2.9) and the 
second-order differentiability of Uj{qj,Cj). Next, let 

(5.84) C(i) ={qeRi: \\q - q*{w{q),p{i))\\ > S} C {q e Ri : q ^ 0} 

where the workload w{q) corresponding to each q G i?^ is defined as in (5.54) and the set 
C{i) is a closed subset of R'l_ due to the first part of Lemma 5.3. Moreover, similar to (5.73), 
we know that h{q, i) < and the equality is true if and only ii q = q*{w{q), p{i))- 

In fact, supposing the if part is true with some q G Q{ki, ...,km) that is defined in 
(2.10), then it follows from (5.83) and the last equality in (5.73) that {pi{i),l ^ ki,...,kjn} 
is the solution to the corresponding optimization problem in (2.5) with {qi,l ^ ki, km} in 
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the associated (J — m)-dimensional utility function. Thus, it follows from Lemma 5.2 that 
{qi,l / ki,...,km} = {ql{w{q),p{i)),l ^ ki,...,km}. Moreover, since q*iw{q), p{i)) > due 
to w{q) > and Lemma 5.2, we know that '4>{Q{t),a{t)) - ip{q*{W{t), p{a(t))),a{t)) < for 
Q{t) = q, which contradicts the fact that q*{w{q), p{i)) is the solution to the cost minimization 
problem in (2.12). Conversely, the only z/part is the direct conclusion of the second part in 
Lemma 5.2. Therefore we have that i) < over C{i). Since h{q, i) is continuous in g 7^ 0, 
we know that there exists a a > such that 

(5.85) h{q,i) < -a in C{i) 

Moreover, since the state space of a(-) is finite, we can consider a as the common constant 
such that (5.85) is true for all i E IC. So the claim in (5.82) is proved. 

Next, we prove that there exists a time T^^^ > for any given e > such that (5.78) is 
true. To do so, we first show that 

(5.86) L{Q{t),a{t)) ^0 as 00. 
As a matter of fact, define 

(5.87) Li(Q(t), a(i)) = L{Q{t),a{t)) - e{t) 
where e{t) is a step function given by 

e{t) = Yl (V'(Q(rn),a(r„)))-V(Q(r-),a(T-))) 

n;r„<t 

(5.88) - J2 (V'(«*(VF(Tn),p(«(Tn)),a(r„))-V'(g*(Ty(T-),p(a(r-)),a(r-))). 

n:Tn<t 

Therefore we can see that Li{Q(t),a{t)) is continuous and bounded over t E [0, 00) since 
Q{t) and p{a{t)) are bounded. Thus we know that e{t) is also bounded over t G [0, 00) due 
to the fact that L{Q{t),a{t)) is bounded. Moreover, since 

(5.89) = < U for a.a. t G [U, 00), 

(Jjlj (J/L 

we know that Li{Q{t),a(t)) converges to some constant as t ^ 00. 

Now, since e(t) is a step function and is bounded, any convergent subsequence of e{t) 
in terms of t corresponds to a sequence of holding time intervals as t — 00 such that the 
convergence of e{t) is true for all t along the sequence of holding time intervals. Moreover, 
since the state space of a(-) is finite, there exists at least one i E K. such that the holding 
time intervals corresponding to this particular state i appear infinitely many times. To be 
convenient, we use [r„, , t^^-^ ) with / E {1, 2, } to denote such a sequence of holding time 
intervals, where r^j is the jump time of a(-) corresponding to the index ni. Notice that 
[Tnp'J'ni+i) with I E {1, 2, } are sampled from a sequence of i.i.d random variables (actually 
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exponentially distributed). Therefore, due to the strong law of large number and without 
loss of generality, we can assume that 

oo 

(5.90) 1] - Tn J = oo. 

1=1 

Therefore, for an arbitrarily given convergent subsequence of e{t), we can obtain a sequence 
of holding time intervals [t„, ,r„j^J (l € {1,2,...,}) with the property (5.90) associated with 
a particular state i G /C. Then it follows from the convergence of Li{Q{t),a{t)) that 

(5.91) L{Q{t),a(t)) ^ Loo > as f ^ oo over i G ^iZi[rn„ rni+,) ■ 

Furthermore, we can claim that Lqo = by showing a contradiction. In fact, if we assume 
Loo > 0, then for any given constant e satisfying < e < L^o, there exists some large enough 
time Ti > such that 

(5.92) L{Q{t),a{t)) > - e > for all te [Ti,oo) n (U;~i[r„^,Tn,^J) . 

Since ■il^{q, i) is continuous and strictly increasing in g G -R+ for each z G /C, it follows 
from (5.92) that there exist some (5 > and o" > such that (5.82) is true for all t G 
[Ti,oo) n (U;~i[T„,,r„;_^J). Thus it follows from (5.87) and (5.89)-(5.90) that 

(5.93) L{Q{t),a{t)) = L(Q(0),a(0)) +e(t) + f dLi{Q{t) ait)) 

Jo at 

Nit)-1 

< C-a (r„;^j -TnJ 

1=1 

< 

for all sufficient large t e [Ti, oo) fl (U;^i[r„p r^j^J) , where N{t) = max{Z : r„, < t} and C is 
a positive constant since e{t) is bounded. However, the derived result in (5.93) contradicts 
the fact that L{Q(t),a{t)) > 0. Therefore the assumption that Lqo > is not true, which 
implies that Lq© = 0. Since the convergent subsequence of a(-) is arbitrarily chosen, we know 
that the convergence in (5.86) is true (readers are also referred to [16] for related discussion 
concerning a continuous Lyapunov function with no jumps.) Hence it follows from (5.86), 
the continuity and strict monotonicity of ip{q, i) in g G for each i e K. that there exists a 
time T^^e > for any given e > such that (5.78) is true. 

Finally, if Q{0) = q* {W (0) , p{a{0))) , then it follows from (5.81) that the claim that 
Q{t) = Q{0) a.s. for all t G [ro,Ti) is true. Hence we finish the proof of the lemma. □ 

5.4 A Key Lemma on Finer Time-Scaling 

It follows from (3.1), (2.17), (2.15), (5.29)-(5.35), and the similar argument as for (5.30) that 

(5.94) W'^i-) =<^ X^(-) + ¥"-{■) 
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where, for each t >0, 

(5.95) r{t) =rJ2 [J%Ms))ds - f]{t) 
which is non-decreasing in i > due to (5.31), (2.16) and (2.20), and 

(5.96) X^{t) = E ^ {E]{t) - S^iT-if))) - pM^))) ds 

^ X{t) as r ^ oo 

where pj(a(-)) = A^(a(-))//ij, and the weak convergence in (5.96) is due to Lemma 5.4, 
Lemma 5.5 and the random change of time lemma (see, e.g., page 151 of [5]) with X{-) given 
by (3.12). Since X{^-) is a continuous process, it follows from the Skorohod representation 
theorem that the convergence in (5.96) can be assumed u.o.c. So, in the rest of this subsection, 
wc will only consider an arbitrarily given sample path for which the above u.o.c. convergence 
holds. 

Now, for a time r > 0, a constant 5 > 0, a large enough integer r, and a fixed time 
r > of certain magnitude to be specified later, we divide the time interval [r, r + 5] into 
a total of [r(5/T] — 1 segments with equal length T/r except the last one, where [•] denotes 
the integer ceiling. The Ith segment with / € {0, 1, \r5/T~\ — 1} covers the time interval 
[r + IT/r, {t + {I + l)T/r) A T]. Then, for any t € [t,t + 6], it can be expressed as 

(5.97) t = T+ {IT + u)/r = rf^^ (n) 

for some I G {0, 1, \r5/T'] — 1} and u G [0,T]. Hence, due to the explanations in (5.27)- 
(5.35), we can define 

(5.98) W^u) = ^W^'"((rV + WT) + ru,a^.,((„)(-)) 

r 

= W''{t) 

for each I € {0, 1, \r5/T~\ — 1} and u € [0, T]. In other words, for each time point, we will 
study the behavior of W^{t) through the fluid process, W^'''{u), over the time interval [0,T] 
(see, e.g., [54] and references therein). Similarly, we can define Q^'^{u) and Y^'\u) through 
Q^{t) and Y^{t). Moreover, define ci and C2 to be the following constants 

(5.99) ci = max(l//Xj) and C2 = ^min(l//^j)^ 
Thus, for any given w > and alH G /C, we have 

(5.100) w < ci\\q*{w, p{i))\\ and \\q*{w, p{i))\\ < C2W 
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since 

In addition, for any e > 0, define 

(5-101) Ti = max. ^T^C2+l)e,ei 2^max{6i,i,6i,2},e/2) rmax{6i,i,6i,2},o-/2} 

where a is determined in Lemma 5.3 and 

(5.102) = (7((c2 + l)e) + e, 62,1 = ci6i,i + e, 

(5.103) 62,2 = max{62,i, + e} + C + e, 61,2 = 6262,2 + e 

where g{-) is defined in (5.76). Then we have the following lemma. 

Lemma 5.7 Consider the time interval [r, t + 5] with r > and S > and suppose that 
there is some constant v >0 such that 

(5.104) lim W''(t) = V and lim QVr) = q* {y, piair))). 
Moreover, let C he an arbitrarily chosen positive constant such that 

(5.105) sup \\X{ti) - X{t2)\\<C with X{-) given by [2,. 12). 

tl,t2e[T,T+<5] 

Then, for any given small enough number e > and a given T >Ti, the following claims are 
true for all large enough r G {1, 2, ...} and all I G {0, 1, \rS/T'\ — 1}: 

(5.106) ||0"''(n) - q*{W'^'^{u),p{a{7]'-'^{u))))\\ < e for all u G [0,T], 

(5.107) WHu) <v + C + 0{e) for all u £ [0,T], 

(5.108) Y'''\u) - y''''(0) = for all u G [0,T] if W^\u) > e for all u G [0,r], 
where lim^-^o 0{e) = 0. 

Proof. For convenience, besides (5.106), we will prove the following stronger claims 
instead of showing (5.107) and (5.108) directly, that is, for large enough r G {1, 2, ...} and all 
nonnegative integers I G {0, 1, \r5/T^^ — 1}, 

(5.109) if W'''\v) <e<C for some u G [0,r], 

then W'^iu) < 62,1, Q'''Hu) < 61,1 for all u G [0,r]; 

(5.110) if W'^iu) > e for all u G [0,r], 

then W'^iu) < 62,2, Q'''\u) < 5i,2, Y''^\u) - Y^^^^O) = for all u G [0,T]. 

Thus the remaining proof of the lemma can be divided into the following two parts. 
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Part One: We justify the claims stated in the lemma to be true when / = 0. As a matter 
of fact, it follows from (5.98) and (5.104) that 



(5.111) 



(Tr^'°(0),Q^'"(0)) = iW'iT),Q'{T)) ^ iu,q*{u,p{aiT))) as r ^ cx). 



Sr,0 



Now, due to the definition of t„ defined in (5.26), we know that [r, r + T/r] C [r„-i, r„) with 
some n G {1,2,...} for all large enough r G {1,2,...}. Thus a{rf'^''^ (u)) keeps some constant 
a(r) for all u G [0, T] when r is large enough. So, as r — )■ oo, we have. 



(5.112) (l^'''^(u),Q^'"(tx)) ^ {W{u),Q{u)) = {u,q*{u,p{a{T))) u.o.c. for all u G [0,r] 

where we have used Lemma 5.5, the uniqueness of the limit, and (5.98). 

Therefore it follows from the first part of Lemma 5.3, (5.112), and the similar argument 
as used in [54] that, for all large enough r G {1,2, ...} and for all u G [0,T], 



(5.113) 



< e 



Thus (5.106) presented in the lemma holds when / = 0. Moreover, it follows from (5.112) 
and (5.105) that the bound estimations in (5.109) and (5.110) are true for all u G [0, T] and 
all large enough r when 1 = 0. In addition, the complementarity in (5.110) can be shown as 
follows. For the given e > in the current lemma, it follows from the first part of Lemma 5.3 
and (5.112) that a cr > can be chosen such that, for large enough r G {1,2,...} and all 
UG [0,T], 



(5.114) 



Q^'°(u)-g*(T^^'°(u),p(a(7?^'°(u))) 



< a 



since a{ri''"'^ (u)) = a(r) for all u G [0,T] when r is large enough. Thus, if W^'' (u) > e for all 
u G [0,r], then 

(5.115) F'^'O(n)-F'^'^O) = f (p(«(r/'-'°(s))) - A,(Q^(r?'-'°(s)), a(r/^'°(s)))) 

j=l JO 
= 0, 

for any u G [0,r], where the first equality of (5.115) follows from (5.95), (5.31) and the 
fact that Q^''^{s) ^ for all s G [0, n] C [0,r] due to the assumption imposed in (5.108), 
furthermore, the second equality of (5.115) follows from (2.11), in addition, the last equality 
of (5.115) follows from (5.13) in the second part of Lemma 5.3. 

Part Two: we prove the claims in the lemma for the case that / G {1, \r6/T~\ — 1} 
by showing a contradiction. As a matter of fact, suppose that there is a subsequence TZi 
of r such that at least one of the claims stated in (5.106) and (5.109)-(5.110) does not hold 
for any r E TZi and some integer I G {1, ...,\rS/T] — 1}, where for later reference, we use 
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Ir € {1, \r6/T~\ — 1} with r € TZi to denote the smallest integer to have such property. 
However, we can show that there is a subsequence 7^2 C TZi such that all the claims stated in 
(5.106) and (5.109)-(5.110) are true for I = 1^ and all large enough r G 7^2- To do so, we first 
construct a subsequence TZs such that (5.106) is true for I = Ir and all large enough r e TZs 
as follows. 

Due to the proof in the first part, we know that the claims stated in (5.106) and (5.109)- 
(5.110) are true for all I G {0, 1, Ir — 1} and all large enough r G TZi. So, for I = Ij. — 1, we 
have 



(5.116) 



Qr,ir-i^Q^ < max{6i,i,6i,2} for all r G 7^l. 



Then we know that {Q'"'''-^(0), r G Tli} has a convergent subsequence from which we can 
find a further subsequence TZ'^ C TZi such that, along r eTZ'^, 



(5.117) 



< 



L - 1 



I Zoo = inf 



h&TZ' 



Ih-l 



< oo 



since < (Ir - l)/r < S/T. Then it follows from (5.97) and (5.117) that, for r G and 

«G [o,2r]. 



(5.118) 



r/'"'''- ^{u) It + looT = rioo as r ^ oo. 



Thus it follows from the definition of Xn defined in (5.26), we know that [r]oo,il^''""~^{u)] C 
[r„_i, T„) with some n G {1,2,...} for all n G [0, 2T] and large enough r eTZ'^. Moreover, due 
to Lemma 5.5, there is a subsequence TZs C TZ'^ such that 

(5.119) iW''^^^-\u),Q'-'^^-\u)) ^ {W{u),Q{u)) with ||Q(0)|| < max{6i,i, 61,2} 
u.o.c. over u G [0, 2r] along TZs. Hence 



(5.120) 



^ 3 + 



Q{u) - q*iWiu),p{air,'''^-\um 



+ 



holds over u G [0, 2T] when r G 7?.3 is large enough, where we have used (5.119) and the first 
part of Lemma 5.3 for (5.120). Then, by (5.78) in Lemma 5.5 and (5.118), we know that, for 
all u G [r, 2T] and large enough r eTZs, 



(5.121) 



e 

<3 



since a{ri'''''"' ^{u)) keeps a constant i G /C for all u G [0, 2r] and large enough r G TZs, and 
moreover, since 



T >Ti> Tniax{6i,i,6i,2},e/2 



36 



where Ti is defined in (5.101). So, for large enough r € 7^3 and u G [0, T], it follows from 
(5.120) and (5.121) that 



(5.122) 



< e 



where we have used (5.120) for the inequality in (5.122). Then we know that the claim in 
(5.106) is true with I = Ij. for large enough r G TZz- 

Next we divide TZ^ into the union of the following two sets, that is, TZ^ = 7?.4 U 7?.5, where 



(5.123) 
(5.124) 



7^4 = {r G 7^3 : W'''''"-{u) < e for some u G [0, T]} 
7^5 = {r G 7^3 : W'^'^^iu) > e for all G [0, T]} . 



Here we remark that at least one of 7^4 and TZ^ must contain infinite numbers. So the 
remaining proof can be divided into the following two parts. 

Firstly, if TZ^ is infinite, then there is a fixed Ur € [0, T] for each r G 7^4 such that 



(5.125) 



Moreover, there is a subset TZ'^^ C TZ4 such that Ur 
u' G [0, r] . Therefore we have 



u' as r — )■ oo for r G TZ'a and some 



(5.126) 



W{0) < W{u') = lim W 



r.lr 



Ur) < e 



where the first inequality in (5.126) follows from the increasing property of W{-), the equality 



in (5.126) follows from (5.119) since W 



Ur 



^/r,ir ^(T + Ur), and the second inequality 



in (5.126) follows from (5.125). Thus we have 

(5.127) 110(0) - q*{W{0), p(a(r/oo)))|| < e 

where r/oo is defined in (5.118) and the inequality in (5.127) follows from (5.119), the first part 
of Lemma 5.3, and the fact that (5.106) is true with I = If for all large enough r G TZ'^^ C TZ^i 
as discussed above. Therefore it follows from (5.127), (5.100) and (5.126) that 



(5.128) 



|Q(0)|| < ||g*(VF(0),/9(a(r/oo)))||+e<(c2 + l)e. 



Then, for all large enough r G TZ'^ and all u G [0, T], we have 



(5.129) 



Q''^^{u) < ||g(n)||+e<6i,i 



where hi^\ is defined in (5.102) and the two inequalities in (5.129) follow from (5.119), the 
similar argument as in (5.122) and Lemma 5.6 respectively. Similarly, for large enough r G TZ'^ 
and all u G [0, T], we have. 



(5.130) 



W''^^{u) < Wiu) + e < ci \\q*{W{u),pia{rioo)))\\ + e < 62,1, 
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where the two inequahties in (5.130) follows from (5.119) and the similar argument as in 
(5.122). Then it follows from (5.129)-(5.130) that (5.109) is true for I = Ir for large enough 
r G 7^4• 

Secondly, if 7^5 is infinite, we can choose a = a{e) as in Lemma 5.3. Then, it follows from 
Lemma 5.6 that, for all u G [0, T], 

(5.131) \\Q{T + u)-q*{W{T + u),p{airjoo)))\\ < \ 

where a(r/oo) = airf '^'^~^{T + «)) keeps a constant i G /C for all u G [0, 2r] and all large 
enough r G 7^5 and moreover, the chosen time T satisfies 

T >Ti> T^ax{bi,i,bi,2},cr/2 

with Ti defined in (5.101). Thus, for all large enough r G TZ^ and all ti G [0,T], we have 



(5.132) Q'-^^^-\T + u)-q*{W'"'^''-\T + u),pia{ri'"^^^-\T + u)))) 



< a 



where the inequality follows from the similar explanations as used for (5.127). Therefore, by 

(5.132) , (5.13) in the second part of Lemma 5.3, and the fact that 

Wr,lr-i^j^ + u) = W'^'^'^iu) > 6 for all u G [0,r], 

we know that Y^''"'^^{T + u) does not increase over u G [0,r] for all large enough r e TZ^, 
i.e., 

(5.133) y'''''^(u) - y''''''(0) = for all u G [0,T]. 

To finish the remaining proof based on (5.133), we need to consider the following two mutually 
exclusive cases for a given large enough r G 7^5 . 

Case One: the condition in (5.110) is true for all I G {0,1,...,/^}- Then we know that 
Y^''"~{u) does not increase over u G [0,T] for all I G {0, 1, ...,lr} due to the induction assump- 
tion and (5.133). So, for large enough r G 7?.5 and all u G [0,T], we have, 

Ir-l 

(5.134) W'^'^^iu) = W'''^{0) + Y,{w'^\T)-W'^\0)) + (w'^^^{u)-W'^^^{0)) 

1=0 

= #^(r) + (l''(r/^'^-(?x)) - 1^(?7'''°(0))) 
< (z/ + e) + (C + e) 

where the second equality in (5.134) follows from (5.94)-(5.95), and the inequality in (5.134) 
follows from (5.104), (5.118), (5.96) and (5.105). 

Case Two: the condition in (5.109) is true for some / G {0,1,...,/^ — 1} and use to 
denote the largest such integer. Then both the condition and the claim in (5.110) are true for 
all I G {ZJ^ + l,...,lr} and therefore the corresponding Y^'^u) does not increase over u G [0, T] 
due to the induction assumption and the discussion as in (5.132)-(5.133). Moreover, by 
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the same discussion as used in (5.118), there is a subsequence TZ'^ C TZ^ such that t]'^'''^ (T) 
converges along r G TZ'^. Thus, similar to (5.134), for large enough r G 7^5 and all u € [0,T], 
we have 



(5.135) 



W'^'^'^ (u) = IT'-'J" (T) + (X" {t]'-'^^ (u) ) - X'' (r?"''™ (T) )) 
< fc2,i + (C + e) 



where the inequality in (5.135) follows from (5.119), the induction assumption (since IIP' < Ir), 
(5.96) and (5.105). 

Therefore, it follows from both of the discussions in Case One and Case Two that, for 
large enough r eTZ'^ and all u G [0,r], we have, 

W^^^^iu) < max{(i. + e) + (C + e),62,i + (C + e)} 



(5.136) 
(5.137) 



< 



^2,2, 



+ e 



where the first inequality in (5.137) follows from (5.122), the second inequality in (5.137) 
follows from (5.100), and the third inequality in (5.137) follows from (5.136). Thus, by 
(5.133) and (5.136)-(5.137), we know that (5.110) is true with I = Ir for large enough r G 7^5. 

In the end, take 7?.2 = 7^4 U TZ'c^ and then we have that (5.106) and (5.109)-(5.110) are 
true in terms of Z = for large enough r G 7^2 C 7?.i . This is a contradiction and hence we 
finish the proof of Lemma 5.7. □ 



5.5 Proof of Theorem 3.1 



As in the proof of Lemma 5.7, our discussion will base on each particular sample path. For 
convenience, we divide the proof into two parts. 

Part One. In this part, we prove the convergence in distribution as stated in (3.10) and 
the related properties (3.11)-(3.13). First of all, since it may be not true that any subsequence 
of {Y'^{t),r G {1,2,...,}} exists a further subsequence that converges to a continuous and 
nondecreasing limit Y{t) when Y'^it) are unbounded (e.g., Y'^{t) = {logr)t). So, we employ 
Lemmas 5.5-5.7 to provide a justification in terms of u.o.c. convergence for {Y'^{t),r G 
{1,2, ...,}}, which can be considered as an supplementary illustration to the corresponding 
claims used in [54], [43], and etc. As a matter of fact, since (0) = for all r G {1, 2, ...}, we 
can conclude that the conditions stated in (5.104) of Lemma 5.7 are satisfied. Moreover, due 
to (5.96), we know that (5.105) is true for an arbitrarily chosen constant C > over any given 
interval [0,T] D [0,ri], where Ti is defined in (5.101). So, by (5.94), (5.98), Lemmas 5.5-5.7, 
we know that, for any t G [0, T] and each large enough rG{l,2,...}, there is a u G [0, T] and 
I G {0, 1, [r(5/r] — 1} such that, for any given small enough e > 0, 

(5.138) < y''(t) = W^u) - X^(t) <C + 0(e) + K, 
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where K is some positive constant due to (5.96) and the continuity of X(t). Therefore we 
know that Y^{t) is uniformly bounded over the given interval [0, T] for all r € {1,2,...}. 
Moreover, since Y^{t) for each r € {1, 2, ...} is nondecreasing and continuous with Y^{0) = 0, 
it follows from the Helly's Theorem (e.g., Theorem 2 in page 319 of [42]) that, for any 
subsequence of these processes, there is a further subsequence TZ C {1, 2, ...} such that 

(5.139) Y''(t) Y{t) for every t € [0,T] along r G 7^ 

where Y(t) is also nondecreasing and continuous with Y(t) = over [0, T]. 

Next, take T G {1,2, ...} and let T — >■ cx) since T > Ti is arbitrarily taken, we know that 
there is a further subsequence TZi C TZ such that the convergence in (5.139) is extended to 
the whole interval [0, oo) along r G TZi, and Y{t) is nondecreasing and continuous over [0, oo). 
Thus, it follows from Theorem 2.15 in page 342, Corollary 2.24 in page 345, and Proposition 
1.17(b) of [29] that the convergence in (5.139) is u.o.c. over [0, oo). Consequently, it follows 
from (5.94) and (5.96) that, along r G T^i, 

(5.140) W''{t) W{t) = X{t) + Y{t) > u.o.c. over t G [0, oo), 

which is continuous in t G [0, oo). 

Thus it follows from (5.139)-(5.140), Lemma 5.5, and the similar argument as used in [54], 
we know that the complementary property as stated in Theorem 3.1 is true. Furthermore, 
take a number § > 0, then for a given e > 0, it follows from (5.106) in Lemma 5.7 that, for 
large enough r G TZi, 



(5.141) sup Q^{t)-q*{W^{t),p{am 

te[o,5] 



or equivalently, for each j G {1, 2}, t G [0, d] and large enough r G TZi, we have 

(5.142) q*{W'{t),p{am - e < Q]{t) < q* {W' (t) , piait))) + e. 

Then it follows from Lemma 5.3 and (5.140) that the following convergence is true (e.g., let 
r — >■ oo first and let e — > later in (5.142)), 

(5.143) g''(t) ^ Q{t) = q*{W{t),p{a{t))) uniformly over t G [0,6]. 

Since S is arbitrarily taken, the convergence stated in (5.143) can be considered true u.o.c. 
over [0, oo). Therefore, we have 

(5.144) {Q''{t),W''{t),Y''{t)) ^ {Q{t),W{t),Y{t)) u.o.c. over [0, oo) along r G 7^l 

with the limit satisfying all the requirements as stated in Theorem 3.1. Consequently, due 
to the uniqueness of solution to the associated Skorohod problem (see, e.g., [11], or [18] and 
[17]), we know that the convergence in (5.144) is true along r G {1,2, ...}. 

Part Two. In this part, we prove the optimality claims stated in (3.14)-(3.15) along the 
line of [54], however, the justification logic and technical treatment are somewhat different. 
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First of all, suppose that all the processes related to an arbitrarily given feasible allocation 
scheme G will be superscripted by an additional G. Moreover, for each t € [0, oo), we define 

(5.145) W^{t) = Uminf W'^'^it) 

which may be infinitely- valued. In other words, for any particularly given t G [0, cx)), there 
exists a subsequence T C {1, 2, ...} such that 

(5.146) W^(t) = lim W'^'^U) along r G T. 

Moreover, let Q denote the set of all the nonnegative rational numbers. Thus there exists a 
subsequence TZ G T such that 

(5.147) W'''^{s) W°{s) along r G 7^ for each s e Q. 

In addition, by applying the similar discussion as in Lemma 5.5, we can select a subsequence 
Tl\ dTl such that, along r G T^i, 

(5.148) f"''^(s) f^{s) u.o.c. over s G [0, oo) as r -5- oo 

where T'-'{s) is Lipschitz continuous and increasing with T^{0) = 0. Furthermore, we can see 
that Q'''^{s), W'''^{s) and Y'''^{s) also converge u.o.c. to Q'-'{s), W^is) and Y^{s) along 
r G 7^1 , which are Lipschitz continuous and satisfy the following relationships 

(5.149) Qf{s) = Xjis) - lijff{s) > for each j G J, 

(5.150) i|.G(,) = ^5M = yG(,)^ 

3=1 

(5.151) y«(s) = ^ (^1^ pj{a{u))du - fj^is)^ , 

where Y'^{s) is nondecreasing with Y^{0) = 0. To further investigate, we define 

(5.152) C = inf {s > : tf (s) ^ Cj{s) for some j € j} 

where Cj{s) is defined in (5.63). Then, under the policy G, it follows from the similar discus- 
sion as in (5.96) that 

(5.153) X'''^{s) X^{s) u.o.c. over s G [0,C) along r G 7^l. 
So it follows from (5.147) that 

(5.154) y'"'*^(s) 7*^(s) along r G 7^l for each s G Q 

where 7*^(5) is some discrete function in s G Q and is nondecreasing since Y'^''-'{s) is nonde- 
creasing for each r G TZi. Moreover, define 

(5.155) Ci = inf {s > : 7^(5) = +00, s G q} , 
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then we know that {Y^''^{s),r € TZi} is uniformly bounded over any compact set of [0, C A^i). 
Thus it follows from the similar explanation as used for (5.139) that there is a subsequence 
C TZi such that 

(5.156) Y'''^{s) Y^{s) for each s G [0, C A Ci) along r G 7^2 

where Y'^{s) is continuous and nondecreasing with Y^{0) = 0, and moreover, it satisfies 

^ ^G^g^ s G Q n [0, C A Ci). 

Then it follows from (5.153), (5.156) and the similar expression as in (5.94) that, along r G 7?.2 
and for each s G [0, ^ A (i), 

(5.157) /3^(s) = lim W'''^{s) = X^{s) + Y^{s) > 0. 

r — ^oo 

However, the complementarity may not be true for [W^ (t) ,Y'^ (t)) . Therefore it follows from 
(5.156)-(5.157) and the minimality of the Skorohod problem (see, e.g., [11], [18], [17], and 
[26]) that 

(5.158) P^{s) > W{s) for all s G [0, C A Ci). 
Hence, if t G [0, A (^i), then we know that, along r G 7^2; 

(5.159) W^{t) = lim VV'^it) = lim W''^^{t) = P°{t) > W{t), 

1 — >oo, r&TZ 1 — ^oo, re7?.2 

which is always true if ^ = Ci = oo- 

Furthermore, if C < Ci or = Ci < oo, and t G [C, C)o), then we can take a time r G [C,i] 
such that ff{T) ^ Cjir) for some j G J. So it follows from (5.149) that rj^(r) < Cj(r) 
and Qfir) > for the j. Then it follows from (5.150)-(5.151) that W^{t) > W^{t) > 0. 
Therefore, along r G 7^2, we have 

(5.160) W^{t) = lim W'^'^it) = lim rW'^it) = +oo > W{t). 

In addition, if ^ > and t G [Ci, oo), then it follows from (5.152) that 

(5.161) liminf Y'''^{t)> lim Y'''°{(^) = = +oo. 

Thus, by (5.153), we know that 

(5.162) W^{t) = lim W'''^{t) = +oo > W{t). 

Since the given time t G [0, oo) is arbitrarily taken, it follows from (5.159), (5.160) and (5.162) 
that the claim (3.14) in the theorem is true for any t >0. 

In the end, it follows from (3.14) and (3.13) that (3.15) is true. Hence we finish the proof 
of Theorem 3.1. □ 
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6 Proofs of Lemmas 4.1 and 4.2 



First of all, to be convenient for readers, we outline the proofs of the two lemmas as follows: 

For the proof of Lemma 4.1, wc first use the optimization technique studied in [25] and [56] 
to characterize the boundary of the MAC capacity region presented in (4.2), i.e., the region 
in (4.2) is convex and thus the boundary of it can be fully characterized by maximizing the 
function J2j=i ^j^j over all rate vectors in the region and for all nonnegative priority vectors 
V = (z/i, z/j) such that J2j=i^j = 1- Then, based on priority vectors and permutation 
schemes, we can determine the number of boundary pieces of the region, which is consistent 
with what is obtained in [34]. Finally, by applying the KKT optimality conditions and the 
implicit function theorem, we can prove that the boundary of the MAC capacity region 
consists of the derived number of linear or smooth curved facets. 

For the proof of Lemma 4.2, we use the duality of the capacity regions between MAC and 
BC to transform the discussion for BC to the one for MAC (see, e.g., [25]). 



6.1 Proof of Lemma 4.1 



Notice that, for a fixed priority vector u, the optimization characterization described in the 
outline is equivalent to finding the point on the capacity boundary that is tangent to a line 
whose slope is defined by the priority vector. Due to the structure of the capacity region, we 
can see that all boundary points of the region arc corner points of polyhedrons corresponding 
to different sets of covariance matrices. In addition, the corner point should correspond to 
successive decoding in order of increasing priority, i.e., the user with the highest priority 
should be decoded last and, therefore, sees no interference. Hence, by [25] and [56], the 
problem of finding the boundary point on the capacity region associated with a descending 
ordered priority vector u can be written as 



(6.1) 

where 
(6.2) 



max f{ri{i),...,rj{i),u) 
{r,(i)>o,Tr(r,(j))<p,je^} 



firi{i),...,rj{i),u) 

J 



z^jlog 




J^j+i)log 



1=1 



which is concave in the covariance matrices. 

Now, let Vj = Vj — I'j+i for j G {1,..., J — 1} and v'j = vj. Then, for any integer 
m G {1, J — 1}, let S{ki, km) denote the following set corresponding to exactly having 
m indices ki, km G {1, ~ 1} such that i'^^ = ... = V}-^ = 0, i.e., 

(6.3)S(A;i,...,fc„) = {f{T^{i),...,Tj{i),u) : Tj{i) > 0,i>j > for j G = ... = = 0> 

J 

kj ^ ki for j / / and j,l € {l,...,m}, lyj > 0, ^i^j = 1}. 
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Moreover, if m = 0, we use S{ko) to denote the set corresponding to v'j > for all j G J, 
i.e., 

(6.4) Siko) = |/(ri(i),...,rj(i),i/) : r,(i) > 0,z>,- > O for j ej,uj> 0,J2uj = l| . 

In addition, ii m = J, uj = 0, we use S{kj) to denote the following set corresponding to 

(6.5) S{kj) = |/(ri(z), ...,Tj(i),u) : r,(z) > for j ej,uj = 0, J^uj = 1 
Eventually, we can define 

^(A;!, ...,km) \i me J, 

(6.6) S{kQM,-,km)=\ S{kQ) ifm = 0, 

, S{kj) if m = J. 

Thus we have 



(6.7) I f{T^{i),...,Tj{i),v) : Tj{i) > 0, i^j > 0, i>j > for j g = 1 

J 

= U U ^(A^o,^!,...,^^). 

m=l ki,...,kmeJ 

Note that the J users can be arbitrarily ordered, so we have J! such priority orders, e.g., 
i^ji > J^j2 — ••• — ^iji where (ji, is a permutation of (1, J). Thus we can see that 

our capacity region is bounded by L boundary pieces with L given by (4.3). In fact, the first 
term J! on the right-hand side of the first equality in (4.3) is the number of boundary pieces 
corresponding to all Uj-^ > > ... > i^jj, Cj{J — j + 1)! {j G {2, J}) is the number of 
boundary pieces corresponding to all = ••• = ^kj with ki,...,kj G {!,..., J} and ki ^ kh 
ioY I ^ h when Ujj > 0, and the last term J on the right-hand side of (4.3) is the number 
of boundary pieces corresponding to = 0. Here we remark that the number of boundary 
pieces obtained through the above method is consistent with the one derived in [34]. 

Next we show the smoothness of these boundary pieces. Without loss of generality, our 
discussion will focus on a specific set <S'(A;o) in a particular user priority order since the 
discussions for all other cases are similar. Therefore, we have that ui > 1/2 > ... > uj > 0. 
Moreover, let y = {yi, ...,y2NNj)' denote the (2iVA'^J)-dimensional vector formed by the real 
part and the imaginary part of entries of ri(i),...,rj(i) in a suitable order. Thus we know that 
f{ri{i),...,Tj{i),u) = f{y,i') is concave in y for each given v > Q. Then the optimization 
problem in (6.1) can be restated as follows. 



(6.8) 



max/(y,i/) 

j/>0 
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subject to 

(6.9) fj{y) = Tr{Tj{i))-Pj<0 for all jej. 

So it follows from the KKT optimality conditions (see, e.g., [35]) that the solution to the 
optimization problem in (6.8)-(6.9) for a function f{y,v) G S{ko) with the associated u > 
can be obtained through the following equations, 

(6.10) yi ^^i^ + j^Vj^^^ =0 foreach ZG{l,2,...,2AriVJ}, 

(6.11) rjjfj{y) = for each jeJ, 

where rjj > for j ^ J are the Lagrangian multipliers. Then our remaining discussion can 
be divided into the following two steps. 

Step One: If there exists some f ^ N such that the problem in (6.8)-(6.9) for the 
function f{y,h') G ^(feo) has at least one optimal solution located in the interior of the 
associated feasible region, where 

(6.12) 

then we have the following discussions. 

Firstly, we suppose that the optimal solution is unique given by y* = (y*, y2ArArj)'- 
Then we know that /(y, v) is strictly concave since it is sufficiently smooth in y for the given 
V due to the definition of /. So it follows from (6.10)-(6.11) that 

(6.13) Fi{y*, v) = ^^^^ = for all I G {1, 2NNJ}. 

oyi 

Moreover, it follows from Theorem 4.3.1 in page 115 of [27] that the following Hessian matrix 

(6.14) vV(2/, ^) = f for ah /, k G {1, 2NNJ}. 

is positive definite at all y within the (2A^A'' J)-dimensional feasible region. Now define 

F{y, u) = {Fi{y, u), I G {1, 2NNJ}}. 

Thus we know that F{y*, i/) = and the Jacobian determinant of F{y, v) with respect to y 
at {y*,iy) is nonzero due to (6.14), i.e., 

(6.15) D{F,,...,F2NNj) ^ Q_ 

D{yi,...,y2NNj) 

Therefore F{y, u) satisfies all the conditions as stated in the implicit function theorem. Hence 
F{y, u) = uniquely determines a (2ArA?^J)-dimensional function y*{i') that is continuous and 
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differentiable with respect to in a neighborhood 0(z^, e) of u. Moreover, (6.15) and (6.13) 
hold in 0(i^, e), which imphes that y*{v) is an optimal solution to the problem in (6.8)-(6.9) 
for each u ^ 0{y,e). 

Secondly, we suppose that the problem in (6.8)-(6.9) for the function /(y, u) G S{ko) has 
multiple optimal solutions located in the interior of the associated feasible region. Without 
loss of generality, we suppose that these optimal points are all in a m-dimensional hyperplane 
that is parallel to each coordinate-axis corresponding to those y with part of its components, 
Vsi where 

y = {vsi eR,le {1, m}, si G {1, 2NNJ}} for some m G {1, 2NNJ}. 

(Here we remark that, if this is not the case, we can employ the method of rotation transfor- 
mation to make this case true.) Therefore, due to (6.2) and the concavity of f{y, u) in y, we 
know that f{y, u) is independent of y^, e y. Thus there exists a (2A?^7V" J — m)-dimensional set 
Py corresponding to each v such that f{y,v) only depends on G (the complement arj^ 
set of 3^) and is strictly concave in those . Therefore for any optimal point y*{i') in the set 
Py and by considering the similar (2NNJ — m)-dimensional problem as in (6. 13)- (6. 14), we 
can conclude that y*{i^) is continuous and differentiable in a neighborhood 0(i/, e) of u. 

So, if the optimal points of f{y, v) are all strictly located within the feasible region when 
V moves in M, it follows from the above discussion that f{y, v) keeps either strictly concave 
or flat with respect to ysi G or G y for all u G TV. Thus we can conclude that all 
the optimal paths y*{y) are continuous and differentiable with respect to G N. In other 
words, any set {r\{u,i), ...,T*j{y^i)} of the optimal covariance matrices is continuous and 
differentiable with respect to G A/". Hence it follows from (4.2) that the corner points of 
the capacity region, which are determined by the following equations, form a smooth curved 
facet f{T\[u,i)^ ...,T*j{v,i)) when v moves in the region A/", and moreover, the facet does not 
depend on the choice of the set {r\{y,i), ...,T*j{u,i)} along u G N. In addition, due to (4.2), 
for all j & J', we have 



(6.16) Cj{u) = log 



^ + E^/«^r(i^,^)i^/(^) 



1=1 



log 



1=1 



However, if some optimal point of f{y,i^) reaches one of the boundaries of the feasible 
region when u moves in Af, then the associated justification for this case is part of the proof 
in the following Step Two. 

Step Tw^o: Without loss of generality, we suppose that f{y, v) is strictly concave for 
all G A/" and otherwise we can employ the similar argument as above. Therefore, if y* = 
(yt, ■■■,y2NNjy is tli^ solution to the optimization problem in (6.8)-(6.9), which is located 
on one of the boundary pieces, and if y* = {y*^, / G {1, m}, si G {1, 2NNJ}} for some 
m G {1, 2A'"iVJ} is the set of components of y*, which are either or on the surface 
fj{y) = for some j & J since fj{y) depends only on part of the components of y, then the 
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remaining components of y* are located in the interior of the corresponding {2NNJ — m)- 
dimensional feasible region and satisfy 

(6.17) w,-..)-^^+i:.«=o 

for each I e C = {I : y*^ e y* \ y*}, where Ci = J Ci {j : fj{y*) = 0} and r]* (j G Ci) are 
the Lagrangian multipliers corresponding to y*. Now let y € B?^^'^ denote the vector whose 
components j/sj for all I G {1, ...,m} are confined in y* . Then f{y,i^) is strictly concave in 
the components of y except those G y* since it is sufficiently smooth in y and since y*^ 
with Z G £ is in the interior of the corresponding {2NNJ — m)-dimensional feasible region. 
Thus it follows from Theorem 4.3.1 in page 115 of [27] that the following Hessian matrix 

vV(y,-)=f^^%^) for an Z,fcG£. 

V ^yi^yk J {2NNJ-m)x{2NNJ-m) 

is positive definite at all y whose components y^, for all I G {l,...,m} are confined in y*. 
Moreover, if we define 



(6.18) F{y,r],i^) = 



Fi{y,r],u) if Z G £, 
Fi{y) = fi{y) if /gA, 



we can conclude that the Jacobian determinant of -F(y, ry, v) with respect to yi {I G C) and 
rjl {I G jO-i) is nonzero at y*. Moreover, due to the definition of /(y, i^) and fj{y) for j G J, 
we know that F{y, rj, u) satisfies all the conditions as stated in the implicit function theorem. 
Hence F(y, r],u) = uniquely determines a (2NNJ + J)-dimcnsional function {y*{v),ri*{u)) 
that is continuous and diffcrcntiable in G A/" (where J is the number of fi {I G Ci) such 
that fi{y*) = 0), and moreover, all the components yl^{i') with / G {l,...,m} are confined 
in y* when ly £ J\f moves. Therefore the remaining proof of this boundary situation can be 
divided into the following three cases. 

Case One: When u e J\f continuously moves to a vector v G M, the optimal point y* {u) 
moves from the interior of the feasible region to the optimal point y*{u){= y*) on the boundary 
of the feasible region. Then we need to prove that y*{u) and its associated derivatives 
converge to y*{iy) and its corresponding derivatives as u converges to u continuously within a 
neighborhood of i/ G in the whole 2A''Arj-dimensional feasible region, which implies that 
the components ysj('u) for all / G {1, ...,m} are not necessarily confined in y* when u e M 
moves. 

In fact, let C2 = {k £ {l,...,m} : 2/5^,(1^) = 0, y*^(z/) G y*} and define the following 
constraints of parallel surfaces, 

(6.19) fjiy, h) = fj{y) - bj = for j G A, 

(6.20) gk{y, b) = - 6^ = for k e £2, 

where b is an arbitrary constant vector whose components are given by bj {j G £1) and b^ 
{k G C2). Therefore, by applying the KKT optimality conditions, the optimal solution y* 
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(6.22) F{y,fi,u,b) = { 



to the the problem (6.8) with the constraints (6.19)-(6.20) should be given by the following 
equations 

(6.21) F,ii',n%.M . '-M^ + S r,-?If^ + ^ «*»^ = 

for each I e C = {I : y*^ & y* \ y*}, where fj* {j G Ci) and f/^ (/c G i22) are the related 
Lagrangian multipliers corresponding to y*. Now, for each y G B?^^-^ , define 

Fl{y,fi,v,b) if / G £, 

Fi{y,b) = fi{y,b) if /G A, 
Fi{y,b) = gi{y,b) if / G £2- 

Then, by the similar argument as used for (6.18), we know that there is a unique {2NNJ + 
J)-dimensional optimal path {y*{u,b),f]*{u,b)) which is continuous and diffcrcntiable with 
respect to {u, b) £ M x R'^ (where J is the dimension of 6), and moreover, all the components 
of y*{u,b) corresponding to y*^ G y* satisfy the constraints (6.19)-(6.20), either being fe; 
{I G C2) or on the boundary fi{y) = bi {I & Ci). Thus we know that y*{u,b) and its 
associated derivatives converge to y*(i^) and its corresponding derivatives as {u,b) converges 
to {iy,0) continuously. Moreover, notice that y*{u) = y*{u,b) when 6/ > (Z G £-2) and 
6/ < (Z G Ci) are all close to zero, which implies that all the components of y*{u,b) 
corresponding to y*^ G y* are also continuous and diffcrcntiable with respect to n G A/" when 
6; > (/ G C2) and 6; < (Z G Ci) arc all close to zero. So we can conclude that y*{u) 
and its associated derivatives converge to y*{i') and its associated derivatives as u ^ v, 
which implies that y* (v) is continuous and differentiable at a neighborhood of v in the whole 
2AriVJ-dimensional feasible region. 

Case Two: When u £ M moves to a vector u G A/", the optimal point y* (u) moves to the 
optimal point y*(z^) (= y*) from a boundary piece of the feasible region next to the boundary 
piece on which y*(z^) is located. The proof for this case is similar to the one as used in Case 
One. Hence we omit it. 

Case Three: When u E Af moves to a vector u e J\f, the optimal point y*{u) moves to the 
optimal point y*{i') (= y*) from a boundary piece of the feasible region that is not next to the 
boundary piece on which y*{i') is located. Due to the concavity of f{y,u), the optimal point 
y*{u) must go first into the interior of the feasible region and then to the other boundary 
piece. Therefore, the proof for this case is the same as the one as used in Case One. 

In the end, we note that the boundary piece corresponding to S{ki, ...,kj) is a J- 
dimensional linear facet, which is determined by the sum-rate capacity bound (see, e.g., 
[56] for more details). Hence we we finish the proof of Lemma 4.1. □ 

6.2 Proof of Lemma 4.2 

It follows from [25] that the capacity region for the J-user MIMO BC with N = 1 and each 
z G /C is given by 

(6.23) nii)=CBc{P,Hii)) 



48 




CMAc{Pi,-,Pj,H^{i)) 



c G : ^ c,- < ^ log / + 5] H]{i)PjHj{i) ,y S C J 



jes ^ jes 



So, due to the similarity of structures between in (4.2) and TZ{i) in (6.23), we can apply 

the similar discussion as for the MIMO MAC and the discussion in [46] to conclude that the 
claims in the lemma are true. Hence we we finish the proof of Lemma 4.2. □ 
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